Beyond Text.
Diffusion from pure noise, multimodal models that see and hear, and generation across code, speech, music and video.
Images from noise, and one model that sees and hears
The player screen is a live animation stage with real diagrams — noise resolves into an image step by step, sound becomes a spectrogram and back again, and one shared space of meaning pulls text, pictures and audio into the same picture as the narration reaches them. Karaoke subtitles in English, fullscreen.
That was the first 10 minutes.
The rest of this chapter — and chapters 2 to 5 — come with lifetime access.
Every idea in the chapter
Each concept the video builds — what it is, what it does, and where it lands in the story. Click any row to open its full breakdown. Filter to find any of them.
| Idea | Category | What it does | In one line | Lesson |
|---|
Beyond text, on one page
Everything Chapter 4 builds, as a reference you can scan: how diffusion creates by first learning to destroy, how a word-predictor learns to see, and how far the single transformer recipe actually stretches.
| Phase | What happens | Who does it |
|---|---|---|
| Forward | take a real photo; add a little noise, hundreds of times, until nothing is left but static | trivial — anybody can add noise |
| Training | show the model images at every noise level; one job — predict the noise that was added | predict it → subtract it → a cleaner image |
| Reverse | begin from pure random noise; remove a little, again and again; shapes emerge, then sharpen | the learned model, step by step |
| Guidance | your prompt becomes a vector, fed in at every denoising step | steers toward an image that matches your words |
Michelangelo: every block of stone has a statue inside it; the sculptor chips away everything that is not the statue. The model frees a picture from the noise. Creation as the careful removal of randomness.
| Control | What it does |
|---|---|
| guidance scale | up = follows the prompt strictly (sometimes too strictly) · down = creative liberty |
| steps | more = higher quality, slower. A trade-off between speed and polish. |
| latent diffusion | compress to a small hidden space, denoise there, expand at the end — why it got fast |
| image to image | start from an existing picture, not noise — a sketch becomes a painting |
| inpainting | mask one region and fill only that — remove an object, change just the sky |
| pose / outline / depth | the prompt says what; these say where and how |
| World | How it is tokenized | Note |
|---|---|---|
| text | BPE chunks | where it all started |
| code | the same — a programming language is a language | often better than text models: strict grammar, and the compiler is an honest judge |
| speech | audio ⇄ text; TTS often uses diffusion | voice cloning from seconds — handle responsibly |
| music | audio into chunks of sound; predict the next chunk | a sentence → a finished track in under a minute |
| images | patches, or latent diffusion | hands and spelling remain hard |
| video | frames + sound | the hardest frontier — consistency and physics |
It is not six breakthroughs. It is one idea wearing six costumes. If you can turn something into a sequence of tokens, you can train a transformer to learn its patterns and generate new examples of it.
Filter by idea, category or lesson.
Chapter 4 · the one sentence: turn anything into vectors and the same machinery can learn it — diffusion creates by reversing destruction, patches let a language model see, and one recipe stretches across text, code, speech, music, images and video.Beyond-text check
Questions and answers reshuffle every load. 70%+ and Chapter 4 is yours.