Replacing 15 tools with a single REPL: the full post-mortem of taking a spreadsheet agent from 50% to 92%
Teaching Coding Agents to do Spreadsheets - Nuno Campos, Witan Labs · Nuno Campos
19 min total·Actually worth watching closely: ~4 min·2 must-watch clips
- 0:17 – 2:10Listen
An underrated hard problem: why LLMs can't handle spreadsheets
Opens by framing the problem: humans see a spreadsheet's structure instantly and visually, while an LLM has to disambiguate layout and meaning on its own, which makes spreadsheets a blind spot for coding agents.
How hard spreadsheets are for LLMs is systematically underrated, and the hard part is structural disambiguation, not the calculation
Purely spoken setup of the problem with nothing on screen to depend on, fine to listen to like a podcast▶ Jump to 0:17Speaker · Nuno Campos - 2:10 – 5:00Listen
The dead-end list: multi-agent and SQL/XML representations all failed
A post-mortem of the failed paths: the multi-agent architecture, SQL, XML and every other single representation. The exploration did leave two byproducts behind, CSV/TSV partial views and an HTML-inspired render engine.
The failed representations weren't wasted; what remained of them became parts of the final architecture
Mostly narrative post-mortem, with nothing in this chapter worth watching, so just listen to why each one failed▶ Jump to 2:10Speaker · Nuno Campos - 5:00 – 7:20Listen
The biggest breakthrough: one REPL with persistent state
Replacing about 15 tools with a single Node.js REPL: variables survive across calls, one call combines several operations, and the agent can interleave reasoning between steps, where before it took 10 or 15 sequential calls that often timed out.
If your agent is making many sequential tool calls, you're really inventing a bad scripting language
The core argument is delivered verbally with no demo on screen, but it's the densest stretch of the talk and deserves full attention▶ Jump to 5:00Speaker · Nuno Campos - 7:20 – 10:50Listen
Building your own verification loop: formula engine plus render engine
Following the pattern where coding agents iterate against a compiler and tests, the team built a formula engine and a render engine as the source of truth driving self-correction, with a warning that a low-fidelity engine, one implementing only half of Excel's formulas, is worse than no loop at all.
The loop is only as good as the engines that power it, and a half-built source of truth turns right answers into wrong ones
Engineering decisions and pitfalls with no live demo, so focus on following the argument for why building it yourself is worth it▶ Jump to 7:20Speaker · Nuno Campos - 10:50 – 14:22Listen
Interfaces go stale, verification loops don't
Separating the two assets: the REPL is the best interface today but may change as the labs push computer use, while four or five model releases showed that more capable models get more out of the verification loop, making the engines the durable asset.
Invest separately in the interface that will go stale and the loop that won't
Strategic reflection delivered purely in speech, good to listen to while mapping it onto how your own project's assets are layered▶ Jump to 10:50Speaker · Nuno Campos - 14:22 – 16:40Skim
Deterministic evaluation and bugs disguised as reasoning failures
Replacing LLM grading with black-box comparison against a golden spreadsheet's inputs and outputs, plus the point that a lot of what looks like the model being dumb is really infrastructure bugs: a wrong example in the prompt followed faithfully, or failing tools that trigger endless retries.
Go deterministic instead of an LLM judge wherever you can, and read the traces to tell 'the model didn't get it' from 'something we can fix'
The benchmark numbers (50%→92%, the jump to 74%) are most likely on slides, so a glance at the data slide alongside listening is enough▶ Jump to 14:22Speaker · Nuno Campos - 16:40 – 18:48Listen
Six generalizable lessons for building agents
The wrap-up: the REPL interface, building your own feedback loop, deterministic evaluation, and domain knowledge narrowing the model's attention, with the emphasis that domain knowledge isn't teaching the model concepts but pigeonholing an all-knowing model onto what the current task needs.
Domain prompts kept improving results across every iteration of the tools and port almost verbatim between approaches
The closing summary depends on nothing on screen, is the best part of the talk to take notes on, and is worth a second listen▶ Jump to 16:40Speaker · Nuno Campos