The problem
Revenue teams already record everything. Every call is transcribed, every email logged, and Monday pipeline reviews still run on gut feel because nobody can read it all. The tax shows up everywhere: CRM stages that flatter the rep, forecasts negotiated in a room instead of derived from evidence, and losses whose causes were audible on a call three months before the deal died.
The brief: given 18 anonymized sales-call transcripts (11 open deals, 3 closed-won, 4 closed-lost), design how AI should turn calls into an operating system for the whole revenue team, and pitch it to RevOps and GTM leadership. It had to be deployable in 30 days on the real stack: Gong, Salesforce, Outreach, BigQuery, Polytomic, Sigma, Slack.
Who it's for
Each stakeholder gets a different product from the same loop. The design treats them separately because adoption dies when it doesn't.
- The CRO buys forecast credibility: a pipeline number derived from call evidence instead of negotiated in a room.
- RevOps owns the system. One source of truth per metric, no new vendors, an audit chain for every number.
- Sales managers get Monday's pipeline review pre-built: the deals where the CRM and the calls disagree, ranked by revenue at risk.
- Reps make or break the rollout. The system drafts their CRM hygiene instead of adding to it, and it never overrides them on forecast. It surfaces disagreements; the manager conversation is the product.
The approach
One loop, closed: transcripts become structured signals, signals land in the CRM and warehouse, dashboards and forecasts read from there, and outcomes feed back to improve the signals. Every design decision exists to keep that loop closed.
Three signal families, fifteen scored measures, each with an anchored 0–3 rubric:
- ICP Fit — does the evidence say this is our buyer?
- Stage Accuracy — does the call support the CRM stage the deal claims?
- Deal Momentum — is this deal moving, or just aging?
One rule was non-negotiable: no score without evidence. Every score cites a transcript quote with speaker and timestamp, so any number in a dashboard can be walked back to the sentence that produced it.
Running the framework over the cohort produced the findings that made the deck land:
- 9 of 11 open deals looked healthy on the ICP × Momentum matrix, yet two of the four recent losses had scored a perfect 15 on ICP fit. Fit wasn't the failure mode. Momentum blindness was.
- Six open deals sat at the wrong CRM stage for the call evidence: three claiming further along than the calls support, three actually further along than the CRM knew.
- One deal got a confident over-call from the extractor. Human review caught it, and it became its own audit slide, with a fix mapped to every layer of the reliability framework.
Architecture decisions
- A multi-stage pipeline instead of one mega-prompt. Per-signal extractors with structured JSON outputs and tiered model routing: cheap models for extraction, capable models for judgment, unit cost visible per deal.
- Buy the plumbing, build the judgment. No new vendors. Gong already records, BigQuery already stores, Polytomic already syncs. The build is the signal layer, the one piece with proprietary value.
- One owner per number. BigQuery holds derived fields, Polytomic is the single reverse-ETL pipe into Salesforce, and Sigma and Slack are read-only. No two systems compute the same metric.
- Six known failure modes, six specific defenses. The reliability layer was designed before scale, rather than after the first incident.
Business impact
Designed as an investment case:
- 30 days to the first closed loop on the existing stack. No new contracts, no data migration. The wedge is one meeting: Monday's pipeline review runs on signal-ranked deals from week one.
- The cohort math is the ROI argument. Six of eleven open deals mis-staged means the forecast was wrong before any model touched it. Correcting stage truth alone changes the number the board sees.
- Loss prevention beats loss analysis. The two lost deals that scored perfect on fit both showed momentum decay in the calls while the CRM said "on track" — months before the loss. Catching one such deal pays for the system.
- Cost scales with call volume, not headcount. Per-deal extraction costs pennies, and the expensive judgment models run only on deals that matter.
Risks & guardrails
The deck dedicated a layer to this, because trust is the actual product:
- Hallucinated confidence is the named failure mode. One extractor over-call was caught in human review and turned into an audit slide, with a defense mapped to every layer.
- Evidence or silence. A signal with no citable quote doesn't ship to a dashboard. Better a gap than a guess.
- AI never overrides reps on forecast. It flags divergence for a human conversation. That one rule keeps the rep constituency from quietly killing the rollout.
- Rubrics are versioned and retrained on outcomes, so scoring drift shows up as a diff instead of a mystery.
Where it goes next
The blueprint is delivered. The natural sequel is the live build: extractors on a real Gong feed, the momentum rubric hardened against a full quarter of outcomes. Either way, the case study was built with the same discipline it proposes — structured extractors writing to disk, evidence-cited outputs, version-controlled frameworks, an audit chain from any slide claim down to the transcript line.