Transformers, ELI5 · Part 8 / 10

How it learns

The weights didn't come from nowhere. The model learned them by playing one game a trillion times — guess the next word, check, nudge.

Where did all those come from? The model learned them — by playing one game, a trillion times.

Guess, check, nudge

Take real text, hide the next word. The model predicts a probability for every word. We know the true one.

Check — the is the negative of the probability it gave the true word: low if that probability was high.
Nudge — tweak every weight a hair to raise that probability.

Which way is “up”? The loss is a hill, and backprop works out the downhill direction for every weight in one pass — then we step a hair that way.

Try it: hit Train step a few times — the real next word climbs, the rest shrink, the loss drops. Billions of these tiny nudges, over the whole internet, are where all the knowledge comes from.

see this in PyTorch

One training step: score the next word, measure how surprised the model was by the real answer, and nudge the weights to be a little less wrong.

import torch
import torch.nn.functional as F

# logits: (seq, vocab) -- one raw score per vocab word, for each position
# targets: (seq,)      -- the id of the word that ACTUALLY comes next
logits = model(tokens)          # forward pass: predict next-token scores
targets = next_token_ids        # the correct answers we want to match

# cross_entropy = average of -log(probability the model gave the TRUE word).
# If the model was confident & right, that prob is near 1 -> -log(1) = 0 loss.
# If it was confident & wrong, the true prob is tiny -> -log(tiny) = big loss.
loss = F.cross_entropy(logits, targets)   # one number: how wrong we are

optimizer.zero_grad()   # clear leftover gradients from the last step
loss.backward()         # work out how each weight nudged the loss (the blame)
optimizer.step()        # nudge every weight a tiny bit to lower the loss

Go deeper: which way is downhill? (backprop)

“Nudge” means stepping downhill on a hill of error. steps the opposite way the slope points; computes that slope for every weight in one backward pass.

Go deeper: how it becomes the assistant you chat with

So far the model just predicts the next word off the internet — it doesn’t answer. Two more training steps turn it into a chatbot: on example questions-and-answers, then preference-tuning — nudging it toward the answers people prefer (the classic recipe is ; many 2026 models use a simpler one called DPO).

Next: why bigger is better — and how to afford it.

Sources · 6

Full bibliography →