Ten training sets of forty noisy dots are drawn from the same hidden rule, and a polynomial of the chosen degree is fitted to each. The ten fitted curves are overlaid with the rule; at a chosen input x₀ the ten predictions form a pile drawn as ticks, with the pile's center and the rule's value marked. Readouts give the bias (center minus truth), bias squared, the pile's variance, the noise variance, and their sum, a simulation estimate of expected squared prediction error at x₀. A second chart shows bias squared, variance and their sum plus noise averaged over all inputs, for every degree from 0 to 20. The observed patterns depend on the sampled training sets. A small sample of unstable fits can have an average far from the truth.

Many datasets, many fits

Ten training sets from the same rule, one fit each. Where the ten fits land at x₀ is a pile — and a pile has a center and a spread.

degree
at x₀: estimated expected error = estimated bias² + prediction variance + σ² 1.00
estimated bias = mean − truth sample mean of predictions truth f(x₀)