Scoring the next word after “the cat sat”. The final vector h is dotted with each word’s output vector to give a score. Softmax, after dividing by a temperature, turns scores into probabilities. Top-k or top-p can drop unlikely words before sampling. Sample many times and the counts follow the probabilities.

the cat sat …

Click a word to see its arithmetic. Each sample is a weighted coin flip, so “on” doesn’t always win; over 100 draws the counts track the percentages.