Turn production logs into replayable test beds, so every fix is proven to help and proven to break nothing that already worked
Continual Learning for AI Agents: From Failures to Durable Improvements - Soheil Feizi, RELAI · Soheil Feizi
23 min total·Actually worth watching closely: ~2 min·2 must-watch clips
- 0:01 – 2:40Listen
Shipping is where it starts
Feizi frames the problem first: during development you have benchmarks and evaluators, and after launch both are gone. The agent makes mistakes in front of real users every day, and nobody turns those mistakes into improvements.
What continual learning has to solve is not that the model is not strong enough - it is that production failures go to waste.
He is essentially standing and laying out the background, with one line of title on screen. Nothing lost if you listen with your eyes closed.▶ Jump to 0:01Speaker · Soheil Feizi - 2:40 – 7:02Listen
Where feedback comes from, and which layer learns
He splits out two fundamental questions: how to get feedback, and how to act on it once you have it. Feedback comes via two complementary paths, automatic log analysis and human expert review; action can land in the model weights, the harness (prompts / tools / workflow), or memory.
Continual learning is not the same as fine-tuning a model - plenty of useful improvements belong in prompts, tools, and memory in the first place.
This is the conceptual skeleton of the whole talk and it comes through in words alone; the layer diagram he draws just restates what he says, so you can follow without watching.▶ Jump to 2:40Speaker · Soheil Feizi - 7:02 – 10:40Skim
Four families of methods, each missing a piece
He goes through SFT / RL / LoRA, trace-to-harness (a coding agent editing the agent directly), prompt search like GEPA, and memory updates, saying where each one gets stuck.
Ranked from expensive to cheap, the cheaper the method the harder it is to verify - memory updates cost the least, but essentially nobody proves they actually worked.
He lays the four families out as a comparison table and walks it cell by cell; a glance at the table gives you the structure, and he talks slower than the table reads, so feel free to speed up.▶ Jump to 7:02Speaker · Soheil Feizi - 10:40 – 14:02Listen
Three requirements of verifiable continual learning
He proposes VCL: a failure has to be replayable as a task you can grade; the repair has to be routed to the layer that actually explains the failure, as the smallest durable change; and improvements have to compound over a lifetime instead of overwriting each other.
'Log plus feedback' is not a learning environment - you need simulated tools, synthetic users inferred from the log, and evaluators that define success before replayability means anything.
The densest stretch of argument in the talk. The screen carries only a few principle names; the real content is in his spoken derivation, so give it your full attention.▶ Jump to 10:40Speaker · Soheil Feizi - 14:02 – 16:50Listen
Do not let new fixes break old skills
The fourth principle, efficiency, with the weight on regression-aware optimization: no regression on past learning environments has to be a hard constraint inside the optimization loop, not a regression suite you run after the change.
The set of past environments only grows, so complexity must not scale even linearly with it - the engineering point he considers hardest in this area.
This is reasoning about constraints, and he says considerably more than the slide shows; it is also the seam where theory turns into product, so hearing it clearly is what makes the demo legible.▶ Jump to 14:02Speaker · Soheil Feizi - 16:50 – 19:10Skim
Two commands into the loop
He switches to the Relai product and shows how two commands attach it to an existing agent, generate learning environments from logs and user feedback, and finally produce a reviewable version update in something like a pull request.
Shipping the improvement as a PR for a human to review is the key design that keeps automatic optimization on a leash - the machine proposes, a person approves.
The screen shows product UI and a flow diagram; following his narration once through is enough. The frames actually worth stopping on come in the hands-on run right after.▶ Jump to 16:50Speaker · Soheil Feizi - 19:10 – 21:02Watch
One loop live: 78% to 97%
A single natural-language description of a scenario generates a complete learning environment; the simulation shows the support agent at only 78%, with two evaluators clearly dragging it down; after a set number of optimization rollouts, the score climbs to 97%.
About 10 points of improvement on average from a single loop - the one piece of empirical support behind all the theory, and worth remembering.
This stretch is all screen work: what the generated environment is configured with, which two evaluators are low on the score panel, how the number jumps after optimization. Audio alone loses half of it - watch the screen.▶ Jump to 19:10Speaker · Soheil Feizi - 21:02 – 22:34Listen
Closing: take these three away
He returns to three key takeaways and the four principles of VCL, restating that learning can happen in layers outside the model, and that every improvement has to be provable and compounding.
If you take away one line: turning production failures into tests you can replay and grade is the price of admission for continual learning.
The summary slide just relists the earlier principles; it is more useful to think through which layer your own agent's production failures live in than to stare at the screen.▶ Jump to 21:02Speaker · Soheil Feizi