Read a Confusion Matrix

See the errors hidden by one score

The previous module's four outcomes - true positive, true negative, false positive, false negative - are usually laid out together in one small table called a confusion matrix: actual classes down the side, predicted classes across the top. For a yes/no problem with labels [0, 1], scikit-learn returns it as [[TN, FP], [FN, TP]] - rows are what actually happened, columns are what the model guessed.

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, predictions, labels=[0, 1])
print(cm)

[[771  49]
 [106  74]]

Reading this: 771 customers were correctly predicted to stay (TN), 49 were wrongly flagged as leaving (FP), 106 who actually left were missed entirely (FN), and only 74 real churners were correctly caught (TP). A single accuracy number can't show you that specific shape - it collapses four very different situations into one figure. If tomorrow's false-negative count doubles while overall accuracy barely moves (because true negatives dominate the total), that's a real and possibly urgent regression a plain accuracy score would hide.

Warning: Always record which axis is "actual" and which is "predicted," and which label (0 or 1) means what. Swapping the two axes, or forgetting which number means "positive," flips the entire interpretation - a mistake that's very easy to make and very embarrassing to explain later.

For anything beyond a one-off notebook check, save the counts as structured JSON rather than a screenshot, so a terminal tool like jq can query it later and a report can be diffed over time:

$ jq '.confusion_matrix' report.json
{"tn": 771, "fp": 49, "fn": 106, "tp": 74}

A useful report also records which dataset and model produced these numbers: the dataset identity and row count, the exact model version, and the time of evaluation - without that context, a confusion matrix is just four numbers nobody can act on with confidence.

Analogy: A final exam score of 80% tells you almost nothing about what the student got wrong. A topic-by-topic breakdown that shows every safety-related question was answered incorrectly tells you exactly what to fix next. The confusion matrix is that breakdown for a model.