The average, and the swing

The average before it happens

Take every value a random quantity X can land on, multiply each value by its probability, and add. That’s the expectation — the average, computed before anything happens:

$$\mathbb E[X]=\sum_x x\,p_X(x)$$

Decoding: $\mathbb E[X]$ is “the expectation of X”. $\sum$ means “add these up”, one term per possible value. Each term is a value $x$ times $p_X(x)$, the probability that X lands exactly on $x$. Every value × its chance, summed.

Picture the distribution as weights on the number line: $\mathbb E[X]$ is its balance point.

A coin scored Heads = 1, Tails = 0: $\mathbb E[X] = 1 \times 1/2 + 0 \times 1/2 = 1/2$.

The coin can never show 1/2. The expectation needn’t be a possible outcome — it’s the long-run average per flip: flip a thousand times, divide the total score by a thousand, and you’ll hover near 1/2.

A four-sided die D, faces 1 through 4, each with probability 1/4. We can write the sum as a table with a totals row — one row per value, and the expectation is the sum of the last column:

value	probability	contribution: value × probability
1	1/4	1/4
2	1/4	2/4
3	1/4	3/4
4	1/4	4/4
sum	1	10/4 = 5/2

Roll and keep rolling. The running average hunts for one number — the expectation. The shaded band is one standard error of the sample mean on either side of the true mean.

Try it

Roll a few times, then keep going. Compare the shrinking variability of the sample mean with the unchanged variability of individual rolls. The sample mean need not move closer on every step or remain inside a one-standard-error band.

For a continuous quantity, swap $\sum$ for $\int$ (an integral): $\mathbb E[X]=\int_{-\infty}^{\infty} x\,f_X(x)\,dx$. Slice the number line into slivers of width $dx$. Each sliver holds a small probability — $f_X(x) \cdot dx$, density × width. Each sliver contributes value × its sliver of probability — and contributions left of zero count negative. Add them all up.

Uniform on [−1, 1] — every point between −1 and 1 equally likely — has density 1/2. Its average is 0 by symmetry: every value has an equally likely negative twin, so the slivers left of 0 exactly cancel the slivers right of 0.

$$\mathbb E[X]=\int_{-1}^1 x\cdot\frac12\,dx=0$$

The swing

The expectation is the center. Variance is the average squared miss from that center:

$$\operatorname{Var}[X] = \mathbb E[(X - \mathbb E[X])^2]$$

Read it inside out: $X - \mathbb E[X]$ is the miss. Why square? Because raw misses average to exactly zero — $\mathbb E[X - \mathbb E[X]] = 0$ is what “balance point” means — so their average can’t tell a tight spread from a wild one. Squaring makes every miss count, big ones hardest; the outer $E$ averages them. For the die, whose center is 5/2, that’s the same kind of table — miss, square it, weight it, total:

value	miss: value − 5/2	squared miss	× probability 1/4
1	−3/2	9/4	9/16
2	−1/2	1/4	1/16
3	+1/2	1/4	1/16
4	+3/2	9/4	9/16
sum			20/16 = 5/4

All four faces have equal probability, but faces 1 and 4 contribute nine times as much to the variance as faces 2 and 3: their distance from the mean is three times larger, and $3^2=9$.

For finite variance, expanding the square gives a useful shortcut: $\operatorname{Var}[X] = \mathbb E[X^2] - (\mathbb E[X])^2$ — the average of the square minus the square of the average. No misses to compute:

• Die: $\mathbb E[D^2] = (1 + 4 + 9 + 16)/4 = 30/4$, so $\operatorname{Var}[D] = 30/4 - (5/2)^2 = 30/4 - 25/4 = 5/4$ — same answer as the table.
• Coin: $\mathbb E[X^2] = 1^2 \times 1/2 + 0^2 \times 1/2 = 1/2$, so $\operatorname{Var}[X] = 1/2 - (1/2)^2 = 1/2 - 1/4 = 1/4$.
• Uniform on [−1, 1]: $\mathbb E[X^2]=\int_{-1}^1 x^2\cdot\frac12\,dx=\frac12\cdot\frac23=\frac13$ — the area under $x^2$ from −1 to 1 works out to 2/3 — so $\operatorname{Var}[X] = 1/3 - 0^2 = 1/3$.

The standard deviation is $\sqrt{\operatorname{Var}(X)}$ — $\sqrt{\phantom{x}}$ is the square root, which undoes the squaring, so the answer is back in the same units as X. That’s the swing’s size on X’s own scale.

Rules for sums

These are rules for expectation and variance. The quantities below have finite means and variances.

Linearity: the expectation of a sum is the sum of the expectations, even when the variables depend on each other. For three fair coin scores, $\mathbb E[X_1+X_2+X_3]=1/2+1/2+1/2=3/2$. This also holds if all three scores record the same flip: then their sum is either 0 or 3, each with probability $1/2$, so its expectation is still $3/2$.

When the pieces are independent — one’s outcome tells you nothing about the other’s — variances add too.

• Three coins: $\operatorname{Var}[X_1 + X_2 + X_3] = 1/4 + 1/4 + 1/4 = 3/4$.
• Die + coin: $\mathbb E[D + X] = 5/2 + 1/2 = 3$ and $\operatorname{Var}[D + X] = 5/4 + 1/4 = 3/2$.
• It even mixes kinds. Coin + uniform on [−1, 1]: $\mathbb E[X + U] = 1/2 + 0 = 1/2$ and $\operatorname{Var}[X + U] = 1/4 + 1/3 = 3/12 + 4/12 = 7/12$.

Linearity of expectation needs no independence. For variance, the full rule is $\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y)$. Independence makes covariance zero; zero covariance is already sufficient, even without independence. We will define covariance in two at once.

Go deeper: where the shortcut comes from

Write $m$ for $\mathbb E[X]$ — it’s just a number. Expand the square: $(X - m)^2 = X^2 - 2mX + m^2$. Take the expectation term by term (that’s linearity), letting the constant $m$ slide out: $\operatorname{Var}[X] = \mathbb E[X^2] - 2m \cdot \mathbb E[X] + m^2 = \mathbb E[X^2] - 2m^2 + m^2 = \mathbb E[X^2] - m^2$. The shortcut follows from expanding the square and applying linearity.

Try the moves

A fresh distribution each round: find the average, work the shortcut in two steps, or add two pieces.

Try it

Some sum rounds refuse to say how the pieces are tangled — on purpose. One of the two answers is still computable; knowing which one is the fine print above.

A probability is an expectation in disguise

Write $\mathbf1_{\{X=2\}}$ for the indicator — a flag that reads 1 when X = 2 and 0 otherwise (the braces hold the condition being flagged). Its expectation follows by considering its two possible values:

$$\mathbb E[\mathbf1_{\{X=2\}}]=1\Pr(X=2)+0\Pr(X\ne2)=\Pr(X=2)$$

The average of a flag is the fraction of the time the flag is up — so any probability can be rewritten as an expectation. Accuracy on a random example can therefore be written as the expectation of a correct-answer indicator. For other loss functions, expected loss is the probability-weighted average of the per-example loss; the loss itself need not be a probability. When a paper writes $\mathbb E_{x\sim p}[\ldots ]$ — the $\sim$ reads “drawn from” — it is asking for exactly this page’s move: the weighted average of whatever sits in the brackets, weights given by p.

Next

Center and spread: two numbers per distribution. Next, four distributions so common they have names.