Why One Neuron Is Not Enough

Some patterns cannot be cut by one line

The exclusive-or (XOR) pattern returns positive when exactly one of two binary inputs is on: (0,0)->0, (1,0)->1, (0,1)->1, (1,1)->0. No single straight line can place both diagonal positives on one side and both diagonal negatives on the other.

x2
1   +      -
0   -      +
    0      1   x1
+ means positive; - means negative

This is not a training failure. It is a representation limit called linear inseparability. Multiple neurons can create several boundaries, and a later layer can combine the regions. That discovery motivates multi-layer networks.

Analogy: One straight fence cannot enclose two diagonal corners while excluding the other two. Several fence segments can form the required shape.

Warning: When a model underfits, training longer does not always help. If the architecture cannot represent the pattern, optimization cannot invent missing capacity.