Once your agent ships, how do you find problems, send PRs, and gate them? Wandero's production practice: a half-hour loop.
The Missing Layer After Launch - Raphael Kalandadze, Wandero AI · Raphael Kalandadze
20 min total·Actually worth watching closely: ~4 min·2 must-watch clips
- 0:00 – 3:30Listen
Why shipping is where the real work begins
Opens with Wandero's production experience: building is the easiest part, agents have no predefined flow and an endless surface, and you cannot write all the conversation paths as tests before you launch.
The post-launch feedback loop is the missing layer, at least as important as the product itself.
Purely spoken argument, nothing on screen — fine to listen to like a podcast.▶ Jump to 0:00Speaker · Raphael Kalandadze - 3:30 – 7:02Listen
The scariest failures are invisible
Walks through silent failures: the agent gets lucky and finds a workaround, marks features complete without checking they worked, finishes the flow but books a different service — no red alerts, technically successful, still failing the task.
Unit tests and rule-based checks only cover one slice; production is what teaches you what you should have tested in the first place.
The examples are all narrated; nothing in this chapter is worth a screenshot.▶ Jump to 3:30Speaker · Raphael Kalandadze - 7:02 – 10:00Listen
The fast loop: three agents in a pipeline
A log monitoring agent runs every fifteen minutes to an hour, digs into the logs to see whether the user ended up stuck, triggers a fix agent to send a PR or a Slack alert, then a separate review agent with fresh context criticizes and gates it.
Once calibrated, problem to PR-ready takes half an hour, and the review agent's independent context is what makes the quality hold.
Mostly architecture explanation with low visual density — just follow how the three roles divide the work.▶ Jump to 7:02Speaker · Raphael Kalandadze - 10:00 – 12:00Skim
Don't let humans be the bottleneck
Agent-written PRs come with diagrams and metadata you can grasp in minutes; the PR agent plus review agent send ten times more PRs per day than three human engineers.
Calibrate the loop until you trust it and you are the only bottleneck — after that, taking the human out is easy.
The PR example around 600s (Mermaid diagrams, metadata) is worth a glance; skim the rest of the argument.▶ Jump to 10:00Speaker · Raphael Kalandadze - 12:00 – 14:04Listen
Zooming out: the session analyzer
The second track handles high-level health: score every conversation, cluster the problems, connect the dots — run it once or twice a week and you can watch across hundreds or thousands of real conversations.
Beyond fixing local problems fast, something has to answer whether the system as a whole is healthy — that is how you keep the feel for it.
A setup section; the concrete payoff is in the live demo at the next stop, so just get the design intent here.▶ Jump to 12:00Speaker · Raphael Kalandadze - 14:04 – 15:50Watch
Live demo: the health dashboard and AI insights
Opens the home-built dashboard on screen: system health, sessions analyzed, cost, plus AI insights giving root cause and a recommended fix for each critical problem, alongside score distribution, sentiment analysis, tool calls, and rejection rates.
The most valuable thing on the dashboard is not the metrics but the AI insights, where the agent connects the dots and finds the patterns.
The live demo starts at 850s with 'let me show you one example'; 873s and 894s were both flagged as frame-worthy — you need to watch the screen.▶ Jump to 14:04Speaker · Raphael Kalandadze - 15:50 – 17:20Watch
Adding the user's perspective: computer use
Beyond the logs-and-code angle, a computer use agent opens the browser, logs in, and simulates a real customer to catch UI problems logs never show; a skill built for their own DOM is much faster than generic browser use, but expect a lot of tokens.
Give the monitoring agents the same access surface a human engineer needs — logs, trajectories, metrics, database, UI, none of them optional.
This chapter has demo footage; the user-perspective test at 982s is the key moment for understanding the equal-access claim.▶ Jump to 15:50Speaker · Raphael Kalandadze - 17:20 – 19:32Listen
Conclusion: the meta harness is the moat
Closes the talk: the moat is not the model alone but the meta harness around it — a system that watches itself, improves, and closes the loop.
Turning post-launch operations themselves into a system that observes and repairs itself is the part nobody can copy.
A spoken wrap-up and takeaway — good to just listen through to the end.▶ Jump to 17:20Speaker · Raphael Kalandadze