Loss Defines What Wrong Means

Training follows the score you provide

A loss function assigns a penalty to predictions. Mean squared error is common for regression and emphasizes large numeric misses. Binary cross-entropy penalizes incorrect binary probabilities, especially confident wrong answers. Categorical cross-entropy handles mutually exclusive multi-class probabilities.

true class = positive
prediction 0.9 -> small cross binary cross-entropy loss
prediction 0.1 -> large loss
prediction 0.001 -> extremely large loss for confident error

The training objective usually averages loss over a batch, sometimes adding regularization. A metric and a loss can differ: F1 may express business value, but its hard threshold makes direct gradient optimization awkward, so training uses differentiable cross-entropy and validation selects thresholds by F1 or cost.

Warning: Optimizing a convenient proxy can produce undesirable behavior. If rare errors matter, class weights, sampling, or a task-specific objective may be needed - followed by evaluation on the actual business metric.

Tip: Ask which mistakes the loss emphasizes, what units it uses, and whether lower values align with real-world value.