AI Foundations · Session 1 · How AI works
A language model doesn't retrieve an answer — it predicts the next token, one at a time. Everything you put in the context window re-weights that prediction. Toggle the context below and watch the dice get loaded.
The model has read the word Java and is about to predict what comes next.
Reading: … Java → what word comes next?
Each block is a real prompting move — a persona, a retrieved doc, a few-shot pattern. They compete to steer the same next token. The winning token glows in the colour of whatever's pulling hardest.
The model attends to everything inside the window. Position matters — but not the way people assume.
Drag the slider down. Tokens don't gently fade with age (that's an old RNN intuition). Inside the window they stay vivid — except the middle, which gets attended to least. The moment a token crosses the window edge its influence drops to zero — a cliff, not a slope. Design implication: put the load-bearing stuff at the start or end of a long prompt, never buried in the middle.
With an empty window the model gives you the internet's average answer — safe, bland, high-probability. Paste in one example and the prediction is dragged toward it: out of the grey centre, into your distribution.
The prompt — same every time
“Draft the opening line of our product launch email.”
The next tokens it predicts default · no context
Nothing about the question changed — only what's in the window. The buzzwords fall away and the answer picks up the doc's plain, human voice. That's the style guide loading the dice.
This is what “out of distribution” means. The grey hump is everything the model has seen a million times — the most likely, most forgettable answer sits right at its peak. Your pasted example builds a new purple peak out in the tail and pulls the prediction onto it. One good example can move the model further than a paragraph of instructions.
Section 3 froze a single prediction. Here's the model actually generating — one token off the belt after another. Generic corporate-speak by default. Paste the style guide and watch the whole stream shift.
Two views of one mechanism. Section 3 shows where the prediction lands (the peak it jumps to); Section 4 shows it happening, token by token. Same move either way — the example in the window quietly re-weights every token the model emits. Nothing is retrieved; it's all prediction, re-tinted by context.