repetition / presence / frequency penalty

Lower the score of words the model has already used, so it stops looping.

logit_i -= penalty for tokens already generated

Temperature and top-k/top-p only reshape the distribution — they don't directly stop the model repeating itself. Penalties do: before sampling, they subtract from the score of tokens already used. A presence penalty applies a flat hit to any word that has appeared at all; a frequency penalty grows with how often it appeared; repetition_penalty is the version in Hugging Face / vLLM / llama.cpp. They act straight on the logits, so they target loops the other knobs can't.

Appears in

From scores to a word — and the loop Transformers, ELI5 · pt 7

Nearby in the brain

e exponential greedy decoding logits min-p sampling probability sampling softmax