Turns quantization, caching, and distillation for diffusion inference into a roadmap you can start on today
You Might Not Need 50 Diffusion Steps — Ziv Ilan, Nvidia · Ziv Ilan
18 min total·Actually worth watching closely: ~6 min·2 must-watch clips
- 0:15 – 2:30Listen
Diffusion is still where LLMs were a few years ago
Opens with the problem: generating an image or a video takes tens of denoising steps, which is slow and expensive, and diffusion's inference-optimization ecosystem is far less mature than the autoregressive one. That sets the through-line of the talk — port over the optimization ideas already proven in the LLM world.
Diffusion inference optimization isn't invented from scratch; it's a migration with signposts, so knowing what happened on the LLM side lets you predict where this goes.
Pure problem statement and setup — the speaker is laying out logic, not showing anything. Listening while you do something else is fine.▶ Jump to 0:15Speaker · Ziv Ilan - 2:30 – 5:00Skim
Quantization: the easiest fruit to pick, rougher on diffusion
Starts from the simplest lever — lowering numerical precision. But diffusion loses image quality after quantization more readily than language models do; using Flux 2 as the example, the speaker explains that it computes the quantization range dynamically at runtime so the range fits the real data distribution instead of being fixed up front.
You can't copy the LLM playbook for quantization on diffusion — the range has to follow the actual data, or quality goes first.
A walkthrough of precision formats and quantization ranges, most likely paired with a diagram on screen. Glance at the slide and follow along by ear; no need to stare.▶ Jump to 2:30Speaker · Ziv Ilan - 5:00 – 7:16Listen
The memory you save is worth more than the time
More on what quantization actually buys: memory drops far enough that consumer GPUs can run it, and inference gets faster. But here's the cold water — diffusion is attention-heavy, so the speedup isn't as striking as on language models.
Quantization's first value is usually fitting the model onto a cheaper card; the speed is a bonus. Don't benchmark it against LLM speedups.
A conclusion-level trade-off between payoff and cost, spelled out in words. Nothing to watch.▶ Jump to 5:00Speaker · Ziv Ilan - 7:16 – 10:00Listen
Caching: don't compute the unchanged part twice
The second line is reusing computation. Whatever barely changes between adjacent denoising steps can skip recomputation, and more modern approaches work at chunk granularity, recomputing only what's genuinely moving — the person walking around gets recomputed, the static background is reused as is.
Caching cuts both ways: set aggressively it visibly hurts quality, so you have to measure the throughput/quality balance yourself — there is no universally optimal setting.
The speaker's spoken analogy — himself moving around versus the static room — carries the whole mechanism. This stretch runs on language, not visuals.▶ Jump to 7:16Speaker · Ziv Ilan - 10:00 – 14:20Listen
Step distillation: distilling steps, not parameters
The heart of the talk. Distillation here doesn't shrink the model; it trains a student to reach 50-step quality in four steps, eight steps, even one — a 10x to 200x performance gain. The speaker compares the two schools, trajectory-based and distribution-based, and mentions real-time video generation already demoed on a single B200 at GTC.
If you want real-time video generation, step distillation is currently the only path that also holds quality; everything else just makes you faster and can't close a gap of this size.
The richest but also the most abstract stretch — the difference in thinking between two families of training methods and their quality trade-offs. Worth putting your work down for; the visuals won't help.▶ Jump to 10:00Speaker · Ziv Ilan - 14:20 – 16:40Listen
The three stack — take them in this order
Pulls quantization, caching and distillation into one roadmap: not three options to choose between, but three layers that stack. Start with quantization, the least work; if that isn't enough add multi-GPU parallelism and caching; and only then take on distillation, the most impactful and the most complex.
Layer them from lowest to highest implementation cost — don't start by chewing on distillation.
A clean ordering recommendation; hear it once and you can apply it straight to your own schedule.▶ Jump to 14:20Speaker · Ziv Ilan - 16:40 – 18:29Skim
Q&A on the compute bar, plus the open-source tooling list
Audience Q&A clears up the biggest worry: distillation needs far less compute than expected — the H100/H200 generation is enough, and small video models need less still. But it's post-training, so build your evaluation first and then decide data strategy. Closes with the open-source tools and pre-quantized weights you can use directly.
General use cases don't need a dedicated dataset, but vertical domains such as protein generation must use in-domain data or the results fall apart.
The ending puts up a list of tool and model names all at once — copying those names down beats listening line by line.▶ Jump to 16:40Speaker · Ziv Ilan