A lecture series · Revised July 2026

The Great Mental Models of Diffusion

A diffusion model learns one narrow skill — undo a small amount of noise — and generation emerges from doing that narrow thing thousands of times in a row. Every idea in this series is an answer to one of three costs that one fact creates: too many steps, no way to steer the process, and too much space to work in.

Thirteen lectures · Six volumes · Primary sources throughout

Scroll slowly ↓

Run the physics backward. Noise is easy to add and, it turns out, learnable to remove — and removing it one small step at a time is more tractable than generating an image in one leap ever was.

The 2020s' generative-image story is really one idea told at increasing resolution: first make denoising work at all, then make it fast, then make it controllable, then make it cheap enough to run on a laptop. By 2025 the same idea started denoising text instead of pixels — the final lecture is where this series rejoins the LLM track.

I

The steps problem

The original process needed on the order of a thousand sequential network calls to produce one image. Sampling was the bottleneck, not training.

II

The control problem

A model that only knows how to undo noise has no notion of "a cat" or "in the style of." Conditioning has to be bolted on, and done carefully or it barely works.

III

The space problem

Denoising a full-resolution image, one pixel at a time, thousands of times, is expensive by construction. Somewhere cheaper to work has to be found.

Volume I

The Reverse of Noise

2015–2020 · learning to undo what you added

1 Learning to Walk Backward The forward noising process, the learned reverse process, and the ELBO that trains it. DDPM's whole idea in one picture. DDPM 2 What the Network Actually Predicts ε-prediction, x0-prediction, v-prediction — the same objective, reparameterized three ways, with different training dynamics. Objectives

Volume II

The Continuous Picture

2019–2021 · noise as a dial, not a schedule

3 The Score Is a Compass The score function points toward higher density. Denoising score matching learns it; Langevin dynamics follows it to a sample. Score matching 4 One Equation, Many Algorithms The SDE view unifies DDPM and score matching as two discretizations of the same continuous process — and hands you a deterministic ODE for free. SDE · ODE

Volume III

Fewer Steps

2020–2023 · paying the sampling cost down

5 Skipping Steps Without Cheating DDIM's non-Markovian reformulation makes the reverse process deterministic — and lets you skip most of the steps for free. DDIM 6 Teaching the Model to Skip Itself Progressive distillation halves the step count generation after generation. Consistency models collapse the whole trajectory to one jump. Distillation 7 Straightening the Path Rectified flow and flow matching replace a curved denoising trajectory with a straight one — fewer steps because there is less path to walk. Flow matching

Volume IV

Steering the Process

2021–2022 · from noise to instruction

8 A Classifier You Don't Need Classifier guidance steers sampling with a separate classifier's gradient. Classifier-free guidance gets the same effect by training one model two ways. Guidance 9 Conditioning on Everything Text embeddings enter through cross-attention. The prompt becomes a set of keys and values the image attends to as it denoises. Cross-attention

Volume V

Making It Affordable

2022–2023 · denoise somewhere smaller

10 Diffusion in a Smaller Room Compress the image with a VAE first, then denoise the compressed latent. Stable Diffusion's entire cost saving in one sentence. Latent diffusion 11 Trading Convolutions for Attention The Diffusion Transformer replaces the U-Net with a plain transformer over patches — and inherits the same scaling laws that built the LLM track. DiT

Volume VI

The 2025–2026 Frontier

where diffusion rejoins language

12 When Diffusion Meets Language Diffusion language models denoise a whole sequence of tokens in parallel instead of writing left to right. The bridge back to Volume I of the LLM track. Diffusion LMs 13 Reading a Diffusion Report The same disclosure spectrum as the LLM track, applied to image and video labs — what a technical report about a generative model does and doesn't say. Synthesis