It's not that the model is bad — the agent has no eyes: a practical checklist for building it verification tools
Your agent is blindfolded — Johan Lajili, Poolside AI · Johan Lajili
10 min total·Actually worth watching closely: ~2 min·2 must-watch clips
- 0:15 – 1:20Listen
Same tools, two different worlds
Opens with a familiar scene: people using AI to code and getting vastly different experiences — some never touch code anymore, others say it produces absolute garbage in their production app. The speaker says the split has nothing to do with which side you're on.
The polarized experience isn't a matter of belief; some people hit a switch and others didn't.
Pure spoken setup — the speaker is standing there making an argument, nothing to see on screen; listen to it like a podcast.▶ Jump to 0:15Speaker · Johan Lajili - 1:20 – 2:35Listen
What you do after "I've got it working"
The real dividing line: when the agent claims it's implemented and all working perfectly, one group concludes on the spot that it's lying and gives up, the other tells it to check those logs, reproduce the problem and try again.
What decides the outcome isn't the model, it's whether you force the agent to close the loop.
The whole segment is reasoning, with no code or diagrams behind it; you lose nothing listening with your eyes closed.▶ Jump to 1:20Speaker · Johan Lajili - 2:35 – 3:50Listen
Dragons in the legacy codebase
Explains why AI looks like a genius on greenfield and a disaster on old projects: on greenfield the agent's intuition is mostly right, while old code is full of dead ends and unexpected structures that it walks straight into.
The difference isn't the age of the project but the feedback loop — the dragons were always there, the agent just can't see them.
The blindfold metaphor is developed in speech; there's no accompanying diagram or case screenshot, so listening beats watching.▶ Jump to 2:35Speaker · Johan Lajili - 3:50 – 5:00Listen
Time for engineers to switch roles
Proposes the shift: spend less time on the product itself and more on making the AI capable of working on it — building verification tools, making the codebase easier to work on, filling out the knowledge bases.
AI multiplies errors as fast as it multiplies speed; without a repaired feedback loop, that velocity is itself a trap.
This is the argument of the talk — the speaker's tone and emphasis are the substance; the visuals add nothing.▶ Jump to 3:50Speaker · Johan Lajili - 5:00 – 6:20Listen
What they built for the agent
Shows Poolside's internal CLI: it lets the agent test a product that lives as a VS Code extension — screenshots, token-compressed page snapshots, extracting backend and frontend logs, restarting services, plus packaged high-level commands like "access this menu" or "send a message and wait for the reply."
Verification has to be broken into primitives the agent can call one at a time, not a vague "go figure out how to look at it."
The tool list is narrated item by item; the CLI is never actually run for the audience, so don't wait for a demo — just write these down and check them against your own project.▶ Jump to 5:00Speaker · Johan Lajili - 6:20 – 7:16Listen
Tailor perception to your own product
Tools should hug the shape of your product: a Unity game might want an ASCII representation of its 3D world for the AI; a system with lots of permissions should let it switch logins in one step. Plus two ways to find what's needed — the places you can see a problem and it can't, and periodically having the AI review your logs for repeated moves.
Make sure the AI is self-served before you work on features; before fixing a bug, require it to actually reproduce the bug once.
Examples like the ASCII world representation are described verbally with nothing put on screen; listening is enough to picture it.▶ Jump to 6:20Speaker · Johan Lajili - 7:16 – 9:42Listen
Q&A: does this replace automated tests?
Answers a key question: primitives like screenshots and logs are quite ephemeral and complement checked-in unit and integration tests. The speaker finds automated tests sometimes too rigid and hard to maintain over time, and prefers mimicking the way a human would actually go and test the application.
The endpoint of self-verification is running multiple agents unattended overnight — without it, that workflow doesn't hold up at all.
The Q&A is dialogue only, the questioner isn't identified, and the camera stays on the speaker — listening is enough, and 1.5x works fine.▶ Jump to 7:16Speaker · Johan Lajili