The chain rule, with numbers

The prediction $\hat y=wx+b$ leads to an error $e=\hat y-y$, which leads to a loss $L=e^2/2$. A change in w affects the loss through those intermediate values.

Keep x = 2, y = 5, and w = b = 1. Then $\hat y=3$, $e=-2$, and $L=2$.

Multiply the rates along one path

Read the path forward first:

$$w\ \longrightarrow\ \hat y=2w+b\ \longrightarrow\ e=\hat y-5\ \longrightarrow\ L=\tfrac12e^2.$$

One local change	Rate at the starting values	Meaning
w changes the prediction	$\partial\hat y/\partial w=2$	A small w change produces twice that prediction change
The prediction changes the error	$\partial e/\partial\hat y=1$	Subtracting a fixed target leaves the rate unchanged
The error changes the loss	$dL/de=e=-2$	Increasing this negative error toward zero reduces the loss

Multiply the rates:

$$\frac{\partial L}{\partial w} =\frac{dL}{de}\frac{\partial e}{\partial\hat y}\frac{\partial\hat y}{\partial w} =(-2)(1)(2)=-4.$$

That is the chain rule. It gives the same −4 we found by expanding the square. It uses the derivatives of the small operations instead of requiring one expanded formula.

For b, only the first part of the path differs: $\partial\hat y/\partial b=1$. The loss derivative is therefore $(-2)(1)(1)=-2$.

Does the notation literally cancel?

The matching intermediate letters can help you remember the order, but the chain rule is a theorem about derivatives. Here the operations are differentiable. Their local linear predictions compose: $\Delta\hat y=2\Delta w$, $\Delta e=\Delta\hat y$, and $\Delta L\approx-2\Delta e$. Substitution gives $\Delta L\approx-4\Delta w$.

Add when a parameter has two uses

Now change the model itself: replace the independent bias b with the same parameter w, so $\hat y=wx+w$. At x = 2 and w = 1, the prediction is still 3.

w now affects the prediction twice: through the multiplier and through the added term. The loss derivative includes both contributions:

$$\frac{dL}{dw}=\underbrace{(-2)(2)}_{\text{through }wx} +\underbrace{(-2)(1)}_{\text{through the added }w}=-6.$$

Holding the second occurrence fixed would give the wrong derivative. A parameter reused in several places must collect the contributions from all of those uses.

Along a path, multiply local derivatives. At a shared input, add the contributions. This is the rule the backward pass will organize.

Explore another chain and a branching example

A separate calculation: the chain begins with u = 3x + 1 and squares u. The branching example uses z = x²(x + 1).

Your turn: one parameter in a product twice

Let $u=w\cdot w$. At w = 3, what is du/dw? Count both uses of w.

Work it through

The first occurrence contributes $1\cdot w=3$ and the second contributes $w\cdot1=3$. Add them to get 6. This agrees with the familiar derivative of $w^2$, namely 2w. A graph must accumulate both contributions even when the input node is shared.

Next: organize these calculations in a backward pass.

Sources and further reading

• Stanford CS231n: backpropagation and local derivatives.
• Baydin et al.: Automatic Differentiation in Machine Learning.
• Full reading list.