Define Success and a Baseline

Improvement needs a reference point

A baseline is a deliberately simple reference system that a proposed model must beat. For classification it might always predict the majority class; for regression it might always predict the training-set median; for operations it might be the current human or rule-based process. Without a baseline, a score has no practical meaning.

Success also has constraints beyond predictive quality. Record acceptable response time, memory and compute budgets, privacy boundaries, explainability needs, frequency of mistakes, and the human effort required to review results. A model that improves accuracy by one point while taking ten seconds per request may fail a 200-millisecond product requirement.

baseline: current rule catches 62% of true incidents
candidate requirement: recall >= 75%, precision >= 50%
operating limits: p95 latency < 200 ms, memory < 1 GiB
safety: no automatic remediation; human approves every page

Analogy: A new route is not better because its map looks sophisticated. Compare its actual travel time, tolls, risk, and reliability against the route people already use.

Note: An offline metric is a proxy for real value. After deployment, verify that better predictions actually improve the intended outcome and do not create new harms or unmanageable work for users.