Post-training · Part 1 / 7

Likely is not good

A pretrained model predicts the most likely next word. That is not the same as being helpful — and the gap is what post-training fixes.

A model trained on a mountain of text learns one thing very well: given some words, predict the likely next word.

That is all the training ever asked for. Success was measured by one question — did it guess the words that actually came next in the text? Nothing in there ever asked the model to be useful.

Likely, not good

So watch what a raw base model does. Ask it:

Explain the moon landing to a six year old in a few sentences.

A helpful answer would be a simple explanation. But the most likely continuation of that line — the thing that follows it most often in text — is another instruction:

Explain the theory of gravity to a 6 year old.

It didn’t answer. It continued the pattern. Ask it to translate “The small dog” to French and it may just keep writing the English sentence — The small dog crossed the road — because that’s a likely next few words.

It isn’t broken. It’s doing exactly what it was trained to do. Likely and good are simply not the same target.

Try it: read each prompt as the base model does — what text usually comes next? — then read it the way you meant it. The gap between the two columns is the whole problem.

Two ways the gap shows up

Not helpful. The model ignores what you meant and just continues the words, like the examples above. The intent of the request never enters the picture.

Not safe. Predicting likely text can mean producing text that is dangerous, false, or toxic — wrong medical advice, harmful suggestions, hateful or stereotyped language. Even a perfectly innocent prompt can lead there, because somewhere in all that training text, such words plausibly followed.

The root cause is one mismatch: the training goal (predict words) is misaligned with what we actually want (helpful, honest, harmless).

Closing the gap

Fixing this is post-training — extra training applied after pretraining. Its job is alignment: bending the model from likely toward good. The rest of this book builds it, in two main moves:

• Show it good answers. Fine-tune on examples of the right response to a request. This is instruction tuning (SFT).
• Teach it from judgments. Let it learn from which of two answers a human prefers. This is preference alignment (RLHF, DPO).

There’s a third lever, separate from both, applied at answer-time rather than during training — we save it for the last chapter.

A base model is fluent. It just doesn’t know what you want yet. Next, we teach it.