temperature

A dial for how risky or safe the model's next-word picks are.

probability_i = softmax(logit_i / T) = e^(logit_i / T) / sum_j e^(logit_j / T)

At each step the model produces a raw score (a "logit") for every possible next word, and temperature is the number you divide all those scores by before turning them into probabilities with softmax. Dividing by a small number (below 1) stretches the gaps between scores, so the top choice dominates and output gets sharp and predictable; dividing by a large number squeezes the scores together, flattening the odds so unlikely words get a real shot and output gets varied and surprising. Because dividing by a positive number never reorders the scores, temperature doesn't change what the model knows or which single word it ranks highest — it only tunes how strongly the model commits to its favorite versus exploring the long tail.

Appears in

From scores to a word — and the loop Transformers, ELI5 · pt 7
Evidence adds points The Math Beneath · pt 3

Nearby in the brain

softmax Bradley-Terry model cross-entropy loss odds reward sigmoid e exponential