principles.fyi · the brain · concept

Bradley-Terry model

Turns two hidden quality scores into the probability one answer beats the other.

P(o_i > o_j) = sigmoid(z_i - z_j)

The Bradley-Terry model assumes each option has a hidden scalar score and that the chance one is preferred over another depends only on their score difference, passed through a sigmoid. So the gap between the two scores is exactly the log-odds of the preference, which means a reward model can be fit to preference pairs by learning scores whose differences match how often people pick each answer. It is the statistical bridge that lets a reward model translate chosen-versus-rejected comparisons into a continuous reward.