500 美元、21 小时 RL 训练，4B 模型工具调用打赢 235B
Stop Making Models Bigger, Make Them Behave — Kobie Crawford, Snorkel · Kobie Crawford
21 min total·Actually worth watching closely: ~6 min·2 must-watch clips
- 0:16 – 4:55Listen
为什么不该一味把模型做大
讲者交代研究动机：业界遇到效果不达标就换更大模型的默认反应，未必是对的答案。同时给出企业选小模型的现实理由——成本、速度、安全，以及金融医疗对本地部署和数据不出境的合规要求。
企业生产用例和个人助手不是一回事，它需要一个被约束住的实现方式，而不是更大的参数量。
This stretch is the speaker talking — nothing on screen you need, so it plays fine as audio.▶ Jump to 0:16Speaker · Kobie Crawford - 4:55 – 7:36Listen
Why not just reach for a bigger model
The speaker frames the target: can we take a smaller model and use RL with the right data to yield the performance gains we are looking for and deliver the application functionality we want? The reasons are cost, speed and security — plus the productionizing question of whether you can keep everything on premise and run the service yourself, without external dependencies or worries about data export and data control, which matters especially for financial data, healthcare and similar domains.
For the specific case of tool use in financial analysis, the claim is that RL is the right tool — not more parameters.
Straight narration setting up the study; listen for the logic, no need to watch the screen.▶ Jump to 4:55Speaker · Kobie Crawford - 7:36 – 9:01Watch
The 235B model going off the rails
A walkthrough of the execution trace for the year-over-year growth of YouTube ads revenue from 23 to 24: the model opened with a query against a table that did not exist, never inspecting the environment to see which tables it could query, came up empty twice, and then hallucinated an answer.
Having the tools is not the same as using them — it could have found out what tables were available and never did.
The evidence is the chain of wrong calls on screen; the speaker is just narrating alongside it, so skipping the visual means skipping the proof.▶ Jump to 7:36Speaker · Kobie Crawford - 9:01 – 12:09Listen
How it was done: expert-in-the-loop data
The first step in getting the smaller model to work was generating a high-quality data set. Snorkel's approach is experts in the loop: an internal platform for the work, with experts solicited per task and topic — PhD-level people for their domains and industry practitioners who know the space, such as financial analysis.
Quality is treated as the core element of the data set, and that emphasis is what the process is built around.
Methodology talk about process and trade-offs; the slide is an outline, so following along by ear is enough.▶ Jump to 9:01Speaker · Kobie Crawford - 12:09 – 15:06Skim
The FinQa environment, published and reusable
The FinQa environment Snorkel built to host these questions provides a specific set of tools and is fully self-contained — everything is built in, no external dependencies sitting in a remote data center you cannot reach. It is available on Prime Intellect infrastructure, saved into the OpenEnv repo on GitHub, and hosted in Hugging Face Spaces by the PyTorch and Hugging Face teams.
The environment is accessible and easy to find, so you can load it up and see how to apply it to your own needs.
The substance here is a list of places to get the environment — pause and capture the links rather than trying to hold them in your head.▶ Jump to 12:09Speaker · Kobie Crawford - 15:06 – 19:11Watch
Side by side: what the 4B model actually does
The correct run, step by step: query for the available tables, inspect the schema with get table info to know how to write the SQL, then run the query — which errors on a revenue column that is not in the table — and self-correct by finding the column it actually needs.
Tool discovery and error self-correction are the real keys to these questions; where the model was failing is not quite intuitive — what it needed to learn was how to use tools, not how to reason better.
All the value is in that execution trace, especially the steps after the error where it fixes the column itself.▶ Jump to 15:06Speaker · Kobie Crawford - 19:11 – 20:34Listen
Wrap-up: rubrics pick the data, RL takes a scalar
Rather than only knowing yes or no at the end, the rubric's richer feedback is used to analyze which behaviors need data sets and to decide which data to work with. The RL cycle itself still gets a single value, as GRPO usually does. The closing slide links to Snorkel's blog post on the study, which in turn links to a partner post from the Agentica team at UC Berkeley.
Fine-grained feedback belongs in the data decision, not in the reward the RL loop consumes.
Closing takeaways with just a link on screen — note the blog post and listen through the rest.▶ Jump to 19:11Speaker · Kobie Crawford