XOR: why one line isn’t enough

A model can fail because its shape is wrong—even when the rule is simple.

1. Two switches. Four answers.

Different → 1. Same → 0. That’s XOR: “one or the other, but not both.” Each input, $x_1$ and $x_2$, is either 0 (off) or 1 (on). The answer is $y$.

2. Try to draw one dividing line.

A simple classifier adds weighted inputs. One side predicts 1; the other predicts 0. Try to put both 1 dots in the shaded area and both 0 dots outside it.

Three out of four is the best any straight line can do. The 1s sit on opposite corners, with a 0 on either side. More attempts won’t fix this model’s shape.

Why no line works

The score is $s=w_1x_1+w_2x_2+b$: weights $w_1,w_2$ multiply the inputs; bias $b$ shifts the boundary. Predict 1 when $s\geq0$, otherwise 0.

Both diagonals meet at $(0.5,0.5)$. A linear score at a midpoint is the average of the two endpoint scores.

• The 1 endpoints would both need scores $\geq0$, so their midpoint score would be $\geq0$.
• The 0 endpoints would both need scores $<0$, so the very same midpoint score would be $<0$.

One point cannot satisfy both. In the widget, Show the diagonals makes the conflict visible.

3. Make two checks, then combine them.

Count the on switches: $x_1+x_2$. At least one on? Good. Both on? Cancel it. Two neurons can make those checks; a final neuron combines their answers.

Each check uses a step: a negative score becomes 0; zero or a positive score becomes 1. Turning a score into yes/no is the nonlinear activation.

Turn off the hidden steps. The two intermediate scores rise together, so their difference stays constant. The network loses its useful shape. Stacking more weighted sums still gives one weighted sum.

The whole network, in three short rules

For the working settings, the hidden outputs are

$$h_1=\operatorname{step}(x_1+x_2-0.5),$$ $$h_2=\operatorname{step}(x_1+x_2-1.5).$$

The predicted answer, $\hat y$, is

$$\hat y=\operatorname{step}(h_1-h_2-0.5).$$

For inputs $x_1=1,x_2=0$: their sum is 1. The first check gives $h_1=\operatorname{step}(1-0.5)=1$. The second gives $h_2=\operatorname{step}(1-1.5)=0$. Finally, $\hat y=\operatorname{step}(1-0-0.5)=1$.

4. What carries over to bigger problems?

The original inputs may be hard to separate. A nonlinear hidden layer can turn them into features that are easier to combine. Here those features are “at least one” and “both.” In larger networks, a feature need not have such a tidy name.

Building a solution is different from learning one. We supplied the settings. Training adjusts weights and biases using errors on examples. Smooth activations let backpropagation guide those adjustments.

Could something other than a neural network solve it?

Yes. For binary inputs, the interaction $x_1x_2$ flags “both on.” This rule gives XOR exactly:

$$y=x_1+x_2-2x_1x_2.$$

For $x_1=1,x_2=1$, the product is 1, so $y=1+1-2(1)=0$. For $x_1=1,x_2=0$, the product is 0, so $y=1+0-2(0)=1$.

A useful new feature can make a simple model enough.