This network has inputs x = (1, 2), target y = 1, two hidden ReLU units, and one linear output. The loss is half the squared prediction error. Next step reveals each forward value, then each backward sensitivity. The text below the graph explains the current step; the table lists all nine parameter values and their loss gradients. The slider changes the top-left weight. At exactly zero, ReLU has no ordinary derivative; the backward pass uses the convention zero.
Two inputs, two hidden units, one prediction. Every displayed value is computed from the current weights. Next step reveals the forward values, then the loss derivatives shown in amber. Here, δ means the derivative of the loss with respect to a node's value.
| parameter | value | ∂L/∂(param) |
|---|
Along one route, multiply the downstream loss derivative by the local derivative. If a value feeds several routes, add their contributions. With the other starting weights unchanged, a top-left weight below 0.3 makes z₁ negative: the derivatives for its incoming weights and bias are zero for this example, even though ∂L/∂h₁ may be nonzero. At z = 0, ReLU has no ordinary derivative; this walkthrough uses zero in the backward pass.