What to expect

A mystery box: 5% of the time it pays $100, 25% of the time $5, the rest — nothing. What is one play worth?

No single play answers that. Most plays pay $0; occasionally one pays $100. The honest question is: what do I get per play, on average, if I keep playing?

Weight each outcome by how often it shows up

Play 100 times and the world roughly keeps its promises: ~5 plays pay $100, ~25 pay $5, ~70 pay nothing. Total ≈ 5×$100 + 25×$5 + 70×$0 = $625. Per play: $6.25.

Shrink that reasoning into one line and you get the expectation:

E[X] = Σ p(x) · x = 0.05×100 + 0.25×5 + 0.70×0 = 6.25

Each outcome, weighted by how often it happens. Not a prediction of the next draw — a property of the whole distribution: the number the running average is forced toward as plays pile up.

Each play changes the running average. With more independent plays, large differences from the expected value become less likely.

Try it

press play once a few times and watch the average thrash — $0, then $33, then $22… Now press play 5,000. The line calms and hugs $6.25. Then drag the $100 probability up and watch the target line move before you’ve played at all: expectation is knowable in advance.

The license to sample

Flip the logic around and you get the most load-bearing trick in machine learning. If the true average is what you want but the distribution is too big to touch — every possible sentence, every possible game — you can sample: draw randomly, average what you see, and trust the long run to drag your estimate toward the truth.

A uniformly sampled minibatch can estimate the average loss over a fixed training dataset. That dataset, in turn, may only approximate the population we care about. Individual batch losses and gradients are noisy; convergence of a sample mean is not by itself a guarantee that an optimization procedure converges or generalizes.

Go deeper: why the average settles

For independent, identically distributed draws with finite variance $\sigma^2$, the sample mean has variance $\sigma^2/n$ and standard deviation $\sigma/\sqrt n$. This is a typical error scale, not a bound that every realized error follows. Reducing that standard deviation tenfold requires 100 times as many draws.

The law of large numbers concerns convergence of averages. It does not, by itself, supply the $1/\sqrt n$ rate without the finite-variance assumptions used here.

Today

Everything downstream optimizes expectations: RLHF’s objective is expected reward under the policy; evaluation scores are expected accuracy over prompts. Whenever a quantity is an average over a distribution, sampling is how it gets touched at all.

Now combine this chapter with the log from chapter one. Weight what by how often it happens? The next chapter picks the single most interesting choice: how surprised you are.

Sources · 2

1. Grinstead, C. M., & Snell, J. L. (1997). Introduction to Probability (2nd rev. ed.). American Mathematical Society. Ch. 6 (Expected Value), Ch. 8 (Law of Large Numbers).
2. Robbins, H., & Monro, S. (1951). A Stochastic Approximation Method. Annals of Mathematical Statistics, 22(3), 400–407.

Full bibliography →