Five kinds of agent hallucination, five code demos run live side by side, all open source and reproducible
Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes, AWS · Elizabeth Fuentes
55 min total·Actually worth watching closely: ~42 min·3 must-watch clips
- 0:00 – 3:18Listen
Hallucination isn't mysterious — it's a context and billing problem
The speaker uses one hotel-booking agent as the running example for the whole talk, starting from token economics: you pay for the words going in and the words coming out, and when what you send is not quite right, too much, or missing something important, the agent starts to hallucinate.
All five techniques that follow are code changes, not prompt changes — that framing is the premise for the rest of the talk
Mostly the speaker talking through the argument over a few explanatory slides; fine to listen to while doing something else▶ Jump to 0:00Speaker · Elizabeth Fuentes - 3:18 – 14:26Watch
Picking 3 tools out of 29 for the model
First the 29 tool definitions are laid out and the cost worked through — each schema is about 170 to 200 tokens, roughly 3,000 tokens per call just for tool descriptions, on every message. Then the agent is changed to search a vector store first and inject only the most relevant tools. Baseline and improved versions run side by side.
If the agent has memory, the whole conversation history is added on top and sent with every single message
The speaker walks through the code section by section and runs both versions to compare token usage and answer accuracy — the numbers and outputs are on screen, so you'll miss half of it without watching▶ Jump to 3:18Speaker · Elizabeth Fuentes - 14:26 – 18:10Listen
How tool swapping actually works, and the managed version
The implementation details of dynamic tool swapping are filled in — because this is an agentic loop, the tool registry lives in the agent state, so you can clear the tools and add the new ones on the next invocation with a few lines of code. Then, why some queries stay ambiguous: with generic dummy tools that have similar names, accuracy has a ceiling.
Token count grows across invocations because the chat history is sent along with the selected tools, but accuracy improves
Largely spoken explanation of the architecture choices, with little on screen you need to watch closely▶ Jump to 14:26Speaker · Elizabeth Fuentes - 18:10 – 30:13Watch
Don't hand arithmetic to vector search
Why averages, counts and multi-hop questions necessarily go wrong with retrieval: the agent only sees the top N chunks of your data at a time, so it cannot aggregate, count or traverse relationships across the full dataset — it estimates and presents the estimate as a real answer. The fix is to have the model write a graph query and let the database compute across all the data.
Vector search always returns something, even when nothing in the store is truly relevant — that is exactly where hallucination breeds
The second half runs the question categories one by one with both approaches' answers and token counts side by side; the Antarctica question in particular only lands if you're watching the screen▶ Jump to 18:10Speaker · Elizabeth Fuentes - 30:13 – 40:18Watch
Put two more agents in charge of watching the executor
A single agent acts and validates its own output in the same loop: when a tool returns an error it doesn't surface it, it rationalizes and writes a confident success response. The fix is three agents in sequence — one acts, one checks, one approves or rejects — with both the code structure and a live run shown.
Self-validation inside the same reasoning loop is no validation at all; there is no separation and no second opinion
The message flow between the three agents and the final rejection are printed step by step on screen — listening alone gets you the conclusion, watching shows you how each error is caught▶ Jump to 30:13Speaker · Elizabeth Fuentes - 40:18 – 44:32Skim
Business rules in code, where the model can't skip them
Rules like payment must be verified before confirmation and a guest limit per reservation are written as Python and registered on the before-tool-call hook, which validates before the tool runs. Three scenarios are compared: confirming a booking without payment, a booking that exceeds the guest limit, and a valid booking for five guests.
Same model, same tools, same prompt — the outcome differs only because the rules are in Python, not in the prompt
Mostly a few requests and block messages in the terminal; the rules themselves are already explained, so a glance at what gets blocked and what passes is enough▶ Jump to 40:18Speaker · Elizabeth Fuentes - 44:32 – 52:48Watch
Don't block the user — steer the agent into fixing it
Two kinds of rules are separated: hooks are all or nothing, they block or approve, which is right for a hard constraint. But a group of six could book two rooms, or a flight is full while the next one has availability — there you want the agent to find an option and complete the task instead of leaving the user waiting. That is steering.
Enforcing rules in code before the tools run is the same pattern Amazon Bedrock AgentCore Policies provides at the infrastructure level, managed for you in production
The same request is run under both strategies end to end, and the deployment architecture at the end needs to be seen to follow where each piece sits▶ Jump to 44:32Speaker · Elizabeth Fuentes - 52:48 – 55:17Listen
One technique per failure mode
The five techniques are mapped back onto the failure modes they answer, plus the resources: everything is in the repo, deployable at once with CDK if you know the Cloud Development Kit, or via a notebook if you're new to AWS, with credits linked so you can deploy for free.
Don't stop at the demo — the point is to take it to production
Purely spoken wrap-up and pointers to resources; noting the repo link is enough▶ Jump to 52:48Speaker · Elizabeth Fuentes