A geometric comparison showing that next-token scoring uses a dot product, not cosine similarity: rotating the prediction direction shows cases where the dot product and cosine similarity pick different winning tokens because the dot product rewards vector length.

Why a dot product, not cosine?

Scoring rewards both alignment and length — so the longer vector can win.

Prediction direction and token vectors in 2D Four token vectors of different lengths and angles, and an adjustable prediction direction arrow. unit circle token vectors (w) prediction direction (h)
10°

Dot product — what the model actually uses

h · w — rewards both alignment and length

Cosine similarity

(h · w) / (‖h‖‖w‖) — alignment only, length divided out