Instruction tuning keeps training a base model with the exact same next-token, cross-entropy objective as pretraining; only the data changes — the target tokens become curated good responses to instructions. A base model continues the prompt's surface form; after tuning on many (instruction, response) pairs the same prompt produces a helpful answer. A template trick converts a plain labeled dataset row into an instruction-response pair.

Instruction tuning

Nothing in the math changes — still predict the next token. Only the target changes: the words to predict are now a good answer to an instruction.

the same prompt — chapter 1

promptExplain the moon landing to a six year old
→
base model
→
model output continues the text

keep training on (instruction → good response) pairs

press instruction-tune — watch the same objective run on better targets

turn a plain labeled row into an instruction
raw dataset row {text: "Loved every minute.",
 label: 1}
→
slot into a template "Loved every minute. Did the reviewer enjoy the movie?" → Yes

the fields become an instruction; the label becomes the good response — just another training pair.

A base model only knows how to continue whatever text it sees, so the prompt’s surface form leaks into the output. Keep training it on many (instruction → response) pairs — same loss, same gradient nudge, the target is now a good answer — and it learns to follow instructions in general, not memorize these tasks.