How the final hidden state becomes a predicted next token: the last token's vector is multiplied by the unembedding matrix to produce one logit per vocabulary word, then softmax turns those into probabilities that are sampled.

Pick the next word

Scores become probabilities — temperature decides how bold the pick is.

Final hidden state multiplied by the unembedding matrix to produce logits Three token rows; the last row is highlighted and feeds into a column of six logit scores, one per vocabulary word. final hidden state [3 × 4] × unembedding [4 × vocab] one score per word

softmax turns those scores into probabilities:

the cat sat … awaiting prediction