Prime Intellect's applied research lead opens up the whole open-source RL stack: the real cost sheet for a $50K full RL run on a frontier model
Building Open Source Agentic Models — Will Brown, Prime Intellect · Will Brown
46 min total·Actually worth watching closely: ~15 min·3 must-watch clips
- 0:13 – 7:15Skim
The open superintelligence stack, end to end
Will Brown introduces Prime Intellect and its full stack across every part of the AI research infrastructure: a GPU compute marketplace at 10,000-GPU scale, the PRIME-RL training framework, the Verifiers environment system and the Environments Hub.
Training open agentic models is a complete vertical stack from compute up to environments, not a single framework.
Around 180s there's a bottom-up stack overview slide worth pausing on; the rest is verbal setup you can move through quickly.▶ Jump to 0:13Speaker · Will Brown - 7:15 – 14:18Listen
The evals flywheel and the environment abstraction
Makes the case that the environment is the unifying unit of post-training infrastructure: one rollout-and-verify loop serving offline evaluation, RL training and SFT data generation.
SFT and on-policy distillation can both be modeled as environment rollouts with a teacher; only the source of the score differs.
A purely conceptual stretch with no key visuals, so it suits listening while you build the mental model for the whole talk.▶ Jump to 7:15Speaker · Will Brown - 14:18 – 21:20Listen
Verifiers V1: separating task from harness
Walks through how Verifiers V1 rebuilds the environment around two independent notions, task and harness, so it can support CLI agents like Codex and Claude Code as well as harnesses you build yourself, with a discussion of group reward design.
The old pattern of tools in a single loop can't accommodate CLI agents or MCP; decoupling task from harness is V1's core motivation.
Mostly spoken reasoning about API design trade-offs with no demo; the point is to follow why the concepts split apart.▶ Jump to 14:18Speaker · Will Brown - 21:20 – 28:21Listen
Trace and Renderers: the tokenization trap
Dissects hidden correctness traps, from many-to-one tokenization to Jinja chat templates quietly stripping newlines, and introduces the new Renderers library and how it maintains a strict two-way mapping between messages and tokens.
Tokenization and chat-template mismatches silently desync the trainer from the inference engine, the most insidious source of bugs in RL training.
A spoken warning about engineering details plus the fix. Nothing you have to watch, just catch the key points.▶ Jump to 21:20Speaker · Will Brown - 28:21 – 35:26Watch
PRIME-RL's async architecture and the cost sheet
Explains why agent settings should give up being fully on-policy (long-tail rollouts hold up the whole batch), presents the thoroughly decoupled client-server async architecture, and gives the measured cost of 1000-step full RL on GLM-5: about 3 days and $50K.
$50K is the same order as many companies' monthly token spend, so large-scale post-training is becoming economically viable for ordinary companies.
The densest stretch in the talk. The architectural decoupling and the cost numbers interlock, so it's worth watching straight through with full attention.▶ Jump to 28:21Speaker · Will Brown - 35:26 – 42:26Skim
Systems optimizations and algorithm decoupling
How PRIME-RL decouples the loss that takes the gradient from the algorithm that prepares the data (a new algorithm is roughly one function assigning advantages), plus the hackability logic behind choosing Torch Titan over Megatron.
This is a research era, and a hackable framework is what lets a team of under 10 people push new algorithms forward in parallel.
Mostly argument, but at 2512s there's a diagram sorting the various post-training algorithms into one coordinate system, worth stopping for.▶ Jump to 35:26Speaker · Will Brown - 42:26 – 46:29Listen
The hosted training platform, and closing
Introduces the hosted training platform: multi-tenant LoRA structurally like multi-tenant inference, billed by the token, with environments developed on your laptop and pushed to the platform. Closes with team hiring.
The recommended workflow separates environment iteration from training infrastructure entirely: develop environments locally, train in the cloud by the token.
A spoken tour of the product shape and roadmap with no demo; catch the workflow advice at the end if it's relevant to you.▶ Jump to 42:26Speaker · Will Brown