Inside the Model.
Text becomes tokens, tokens become geometry, attention decides what matters — and the Transformer falls out.
Tokens, meaning, and the mechanism that changed everything
The player screen is a live animation stage with real diagrams — sentences shatter into tokens, tokens fall into a space where meaning becomes distance and direction, attention draws its weights live between words, and the Transformer assembles itself block by block as the narration reaches each part. Karaoke subtitles in English, fullscreen.
That was the first 10 minutes.
The rest of this chapter — and chapters 2 to 5 — come with lifetime access.
Every idea in the chapter
Each concept the video builds — what it is, what it does, and where it lands in the story. Click any row to open its full breakdown. Filter to find any of them.
| Idea | Category | What it does | In one line | Lesson |
|---|
The whole machine, on one page
Everything Chapter 2 builds, as a reference you can scan: how text becomes tokens, how tokens become meaning, how attention decides what matters, and how the transformer stacks it all into the engine behind every model you have used.
| Step | What happens | Result |
|---|---|---|
| start | vocabulary = individual letters only | every word spelled out one character at a time |
| merge 1 | most common pair is t + h | th becomes one token |
| merge 2 | most common pair is now th + e | the becomes one token |
| … × 1000s | repeat, thousands of times | common words = 1 token · rare words = several pieces |
| result | the vocabulary (~50k–100k in most models) | common things get short · rare things stay flexible |
| Role | The question it answers | In the meeting-room analogy |
|---|---|---|
| Query (Q) | "who here is relevant to me?" | the question you walked in with |
| Key (K) | "here is what I am about" | the name tag each person wears |
| Value (V) | "what I hand over, if chosen" | what they actually tell you |
| dot product | how strongly does Q match K? | scanning the tags for relevance |
| softmax | turn raw scores into weights summing to 100% | deciding how much to listen to each |
| the blend | new meaning = weighted mix of the values | you leave with a combined answer |
| Sum | Lands on | What it proves |
|---|---|---|
| king − man + woman | queen | a clean direction for gender emerged — nobody defined it |
| Paris − France + Italy | Rome | a direction meaning "the capital of" — it was never taught geography |
Filter by idea, category or lesson.
Chapter 2 · the one sentence: text becomes tokens, tokens become points in a space of meaning, attention lets those points read each other, and a stack of identical blocks turns that into one very good guess at the next token.Inside-the-model check
Questions and answers reshuffle every load. 70%+ and Chapter 2 is yours.