9 minutes on why "can you verify" is replacing "can you code": a hands-on retrospective of building Vector Harness.
Your coding agent doesn't always follow your rules — Talha Sheikh, Checkout.com · Talha Sheikh
10 min total·Actually worth watching closely: ~3 min·2 must-watch clips
- 0:15 – 2:00Listen
The pain: "Task completed" isn't trustworthy
Opens from the everyday scene: the agent reports done, but something has failed or a small piece is missing, and you're forced to be the only enforcement layer.
The real issue is trust — you can't trust the agent to have actually finished the task.
Purely spoken setup of the problem, no key visuals. Fine to listen to on a commute.▶ Jump to 0:15Speaker · Talha Sheikh - 2:00 – 3:30Watch
Vector V1: deterministic checks via hooks
How Vector Harness works: Claude hooks fire when each session finishes and run the deterministic tests defined in a config file; failures go back to Claude, which retries until everything passes.
Verification should be a deterministic automated loop, not a human re-checking by hand.
137.88s is the LLM-picked key visual moment; the mechanism is far faster to follow with the screen than by ear.▶ Jump to 2:00Speaker · Talha Sheikh - 3:30 – 5:15Listen
Enforce, don't instruct
Argues that the best spec, the best MCP servers and all the context engineering you want are still only instructions — instructions are not verification, so the check has to be enforced the moment the agent claims it's done.
Better prompts won't fix reliability; enforcement will.
Mostly argument, no strong visual dependency — good for hearing the slogan and remembering it.▶ Jump to 3:30Speaker · Talha Sheikh - 5:15 – 6:25Watch
The harder the guardrails, the smaller the model
With deterministic guardrails in place you can use a Haiku or even an open source model instead of an expensive frontier one for the same task, and stack async tasks on top to spread the cost further.
Time invested in the harness comes back as an order-of-magnitude drop in inference cost.
325.51s is the LLM-picked value moment; the cost comparison is more direct with the slide in view.▶ Jump to 5:15Speaker · Talha Sheikh - 6:25 – 7:18Listen
Rebuttal: more capability ≠ more reliability
Answers the challenge that smarter models make the enforcement layer redundant: a new model increases capability, and capability doesn't automatically buy reliability.
Don't count on the next model generation to solve reliability for you.
A short spoken rebuttal, no visual dependency.▶ Jump to 6:25Speaker · Talha Sheikh - 7:18 – 8:30Skim
State of play: everyone builds their own, none of it transfers
Uses Qodo's PR review feedback loop as an example to show that not trusting agent output by default is now industry consensus — yet every enforcement scheme is private and fragmented, and hard to standardize.
A harness has to include verification, and that layer has no common standard today.
At 448.15s the speaker points at the company landscape on the slide ("over here") — a glance at the graphic is enough, no need to watch line by line.▶ Jump to 7:18Speaker · Talha Sheikh - 8:30 – 9:48Listen
Closing: value moves to the verification you design
Wraps up: however fast agents produce code, the delivery bottleneck is the verification layer, and an engineer's core value is shifting from the code we create to the verification we design.
The question isn't "can you code" but "can you verify?"
A summing-up spoken statement — remembering that one line about the shift is enough.▶ Jump to 8:30Speaker · Talha Sheikh