One claims review case, walked end to end through the four steps from requirements to production
AI System Design: From Idea to Production - Apoorva Joshi, MongoDB · Apoorva Joshi
29 min total·Actually worth watching closely: ~10 min·3 must-watch clips
- 0:02 – 4:10Listen
What gets hard once AI can write the code
The opening argument: vibe coding works great for low-stakes things you can eyeball for correctness, but the moment a system has other people depending on it, the work shifts to defining the requirements, the system design and the evaluation criteria so your coding assistants build the right thing. She notes this is the same conclusion recent talks from Anthropic and OpenAI have converged on.
"Specs are the new code" — the bottleneck moved from writing code to saying clearly what you want.
Mostly a spoken argument over a title slide; nothing is lost listening on your commute.▶ Jump to 0:02Speaker · Apoorva Joshi - 4:14 – 8:31Watch
Write the business problem first, then gather the constraints
Enters the health insurance claims review example that runs through the whole talk. She demonstrates how to write the problem as one sentence that names a specific user, quantifies the pain point and says nothing about whether it'll be an agent, then walks the constraints one by one: regulatory compliance, where patient data is allowed to live, which model vendors are approved for procurement, which cases require a human signature, and the service-level expectations on latency and budget.
Constraints aren't a checklist you run after designing — they shape every downstream architecture decision, so you need them first.
Both the sample problem statement and the constraint list appear as full pages of text while she reads only the highlights; listening alone loses the exact wording on screen, which is the part most worth taking away.▶ Jump to 4:14Speaker · Apoorva Joshi - 8:35 – 14:05Watch
From a data inventory to a derived architecture
Design starts with data: what data you need, where it resides, whether it's consumable in its raw form, and running the pipelines at the rate each source updates. Then processing and retrieval matched to the data — long documents chunked, embedded and tagged with metadata for vector or hybrid search, with medical terminology handled by metadata filtering and a keyword fallback; structured records just get an exact match with PII removed. Finally she maps a claim request end to end, and the architecture falls out of it.
Don't start by building an agent, and don't let a coding assistant pick your architecture and stack; map the request flow first and derive the simplest system that meets the requirements.
The request flow diagram in the second half is the densest single screen in the talk — the branch conditions and the final write-back are drawn on it, and she doesn't read every line out loud.▶ Jump to 8:35Speaker · Apoorva Joshi - 14:10 – 17:28Skim
Choosing among four design patterns
Runs through the available patterns and what each is good for in the claims system: retrieval augmentation for external knowledge, control flow so the model only performs certain tasks inside a predetermined workflow, the model as a classifier-router that only dispatches, and human review inserted at the critical points. Her position is to combine what you need rather than defaulting to a fully autonomous agent.
Patterns are there to be assembled; the constraints listed earlier decide the selection, not which one is trendiest.
The visuals are box-and-arrow pattern sketches that add no information of their own — one glance to see which pattern maps to which step is enough.▶ Jump to 14:10Speaker · Apoorva Joshi - 17:31 – 20:51Listen
Let the system explain itself, let reviewers feed it back
On the interaction layer: citations let the system explain itself by quoting the guidelines and policies behind each verdict, while the real users — the claims reviewers — get a way to override a verdict or flag a hallucinated citation, so their normal work produces the improvement signal. The back half moves on to stack selection.
The feedback channel should grow out of what reviewers already have to do, not be a separate rating form bolted on.
Spoken product judgment over a bullet list that doesn't change — listening is more efficient than watching here.▶ Jump to 17:31Speaker · Apoorva Joshi - 20:56 – 23:57Skim
Evaluation and guardrails belong in from day one
Because LLM-based systems are probabilistic, guardrails became necessary in this era: irrelevant inputs get rejected on the way in, responses without citations get caught on the way out, and the guardrail compliance rate itself has to be quantified. Metrics come in three layers: response quality (is the answer faithful to what was retrieved), a domain north star metric (claim processing time) and system health (token cost).
Evaluation isn't a gate you add before shipping — it gets built alongside the system; you can't improve what you can't measure.
The metrics are laid out in a table by layer; scanning the table is much faster than hearing her read each row, so it's worth pausing for a screenshot.▶ Jump to 20:56Speaker · Apoorva Joshi - 24:01 – 28:46Listen
Beyond accuracy: cost, latency, reliability
After shipping you can also track implicit signals like how often a human overrides the AI verdict and how long a review takes. Optimization splits three ways: accuracy is really about optimizing the information that reaches the context window (prompts, reranking, memory), cost and latency come from semantic caching and batch processing, and reliability comes from structured outputs that always return a fixed shape with the verdict and its citations. She closes by restating constraints first, simplest design, evaluation built in, and points to the resource link.
"Accuracy looks good" doesn't mean ready to ship — cost, latency and reliability are three non-negotiable gates.
The optimization techniques are listed out loud one by one, so listening is fine; only look up in the last few seconds to catch the link on the resources slide.▶ Jump to 24:01Speaker · Apoorva Joshi