Probabilities Become Decisions

A score needs a cost-aware policy

A classifier's probability estimate does not choose an action by itself. A decision threshold converts it into a class: probabilities above the threshold trigger the positive action. Lowering the threshold usually catches more true positives and creates more false positives; raising it usually does the reverse.

Expected decision quality depends on costs and benefits. Missing a security incident may cost far more than reviewing a false alarm, justifying a lower threshold. Automatically blocking a customer has a high false-positive cost, demanding stronger evidence and perhaps human review.

score 0.72
threshold 0.50 -> alert
threshold 0.80 -> do not alert
same model, different operating policy

Warning: A number labeled 0.8 probability may not be calibrated. Calibration means cases assigned about 0.8 actually occur roughly 80% of the time. Ranking can be good while probabilities remain overconfident.

Note: Thresholds belong in versioned policy and evaluation, not hidden in application code as an unexplained 0.5.