Derivatives Measure Local Change

Which way and how steep?

A derivative measures how an output changes when one input changes by a tiny amount. On a graph it is the local slope: positive means the curve rises as you move right, negative means it falls, and near zero means locally flat. ML uses derivatives to ask how changing a parameter would change the model's error.

loss
 ^        /
 |      /
 |    _/  <- slope near zero at the valley
 +--------------> parameter value

For many parameters, the gradient is a vector containing one partial derivative per parameter. It points toward the steepest local increase in loss. Moving in the negative-gradient direction therefore heads downhill fastest for a sufficiently small step.

Analogy: In fog on a mountain, you cannot see the whole landscape. You feel the slope underfoot in every direction and take a cautious step downhill. The gradient is that local slope compass.

Note: A derivative describes nearby behavior, not a complete map. Complex loss surfaces can contain flat regions, narrow valleys, and saddle points where local guidance is imperfect.