principles.fyi · the brain · concept

in-context learning

The model picking up a new task from examples in the prompt, without any training.

examples in prompt -> model infers the rule -> applies it to new input (weights unchanged)

In-context learning is the striking ability of a large model to learn a pattern purely from examples placed in its context window — no weight updates, no gradient descent, just reading the prompt and continuing it. It is the mechanism that makes few-shot prompting work, and Book 01's induction heads are part of how it happens: the model spots a pattern that started earlier and completes it the same way. The 'learning' is temporary and lives only in that one conversation; close the chat and it's gone.