Why Nonlinearity Matters

Stacked lines are still one line without activation

If every layer only multiplies by a matrix and adds a bias, several layers collapse mathematically into one linear transformation. Depth would add computation without adding new shapes. A nonlinear activation function between layers prevents that collapse and lets networks form curved, piecewise, and interacting decision regions.

linear(linear(x)) = another linear function
linear -> nonlinear activation -> linear = richer function

For XOR, hidden neurons can detect regions such as at least one input is on and both inputs are on; the output combines those intermediate signals to recognize exactly one. Nonlinearity makes those region-building steps possible.

Scenario: An engineer builds ten stacked dense layers but accidentally omits every activation. Training works, parameter count grows, and performance remains equivalent to a single linear model because the whole network still represents one flat transformation.

Warning: More layers alone do not guarantee a more expressive network. The operations between them determine what function family the network can represent.