principles.fyi · the brain · concept

mean (average)

Add the numbers, divide by how many: the balance point.

mean = (x1 + x2 + ... + xn) / n

The mean is what each value would be if everyone shared equally. Picture the numbers sitting on a seesaw: the mean is the exact spot where it balances, with the highs and lows canceling out. Layer norm finds this balance point for a vector and slides everything over so the new center sits at 0 — a fair, even starting line.