The running total, and its slope

A distribution tells you where probability is located. Its cumulative distribution function, or CDF, tells you how much has accumulated up to a chosen value.

The running total

F_X(x)=\Pr(X\le x).

The capital X is the random variable. The lowercase x is the threshold you choose. For example, $F_X(2)$ asks for the chance that X is at most 2.

As you move the threshold to the right, you include more outcomes. So the CDF cannot decrease. Far enough to the left its limit is 0; far enough to the right its limit is 1. For unbounded distributions these are limits, not values reached at a finite endpoint.

A staircase and a ramp

For the number of heads in three independent fair coin tosses:

x	probability at x	probability up to x
0	1/8	1/8
1	3/8	4/8
2	3/8	7/8
3	1/8	1

Between these values the CDF is flat. At each value it jumps by the probability sitting there. At the jump itself, use the upper value because “at most” includes the endpoint.

For a uniform draw on [−1,1], the CDF is a ramp between the endpoints, with flat sections outside:

F_X(x)=\begin{cases} 0,&x<-1,\\ (x+1)/2,&-1\le x\le1,\\ 1,&x>1. \end{cases}

Inside the interval, x+1 is the length collected so far; divide by the total length 2. At x=0, the CDF is 1/2.

Questions become subtraction

For any distribution,

\Pr(a<X\le b)=F_X(b)-F_X(a).

Subtracting $F_X(a)$ removes everything at or below a, including any probability exactly at a.

For the coin count, the chance of one or two heads is $F_X(2)-F_X(0)=7/8-1/8=3/4$ . Subtracting $F_X(1)$ instead would remove the one-head outcomes too.

For a variable with a density, single points have probability zero, so including or excluding an endpoint does not change the answer. For the uniform draw, $\Pr(0\le X\le1/2)=3/4-1/2=1/4$ .

The slope of the running total

Move from x to $x+\Delta x$ , with $\Delta x>0$ . The increase in the CDF is the probability collected in that short interval. Divide the increase by its width:

\frac{F_X(x+\Delta x)-F_X(x)}{\Delta x}.

This measures probability per unit of x. Where the CDF is differentiable, the limit as the width approaches zero is its derivative. For the smooth densities used here:

f_X(x)=F_X'(x)=\frac{dF_X}{dx}.

The lowercase f is density; the uppercase F is accumulated probability. A tall density means the total is increasing rapidly.

For our uniform ramp, F rises by 0.05 over a width of 0.1: the slope is 0.5. For a non-flat density, density times a small width is an approximation:

\Pr(x<X\le x+\Delta x)\approx f_X(x)\Delta x.

A discrete CDF instead has derivative zero between its jumps and is not differentiable at the jumps. Its ordinary derivative cannot recover the probability mass; use the jump sizes.

Adding the slivers back

Divide an interval into narrow strips. A strip near x has width $\Delta x$ and approximate probability $f_X(x)\Delta x$ . Add the strips and take the limit as they become finer. That limiting sum is an integral:

\Pr(a\le X\le b)=\int_a^b f_X(x)\,dx.

The bounds a and b say where to start and stop; dx says that the slices run along the x direction. For a continuous density this area equals $F_X(b)-F_X(a)$ .

This is the useful connection between differentiation and integration: the density gives the rate of increase of the total; integrating the density accumulates the total.

Compress a distribution into its mean and variance.