Leakage and Data Contracts

Do not let the answer sneak into the question

Target leakage happens when a feature contains information that would not exist when the prediction is made, or directly reveals the target. An exit-survey score recorded after cancellation can make training scores look spectacular while being useless for predicting future cancellation.

Ask a time-based question for every column: Could the application know this value at the exact prediction moment? Also inspect suspicious correlations, duplicated identifiers, aggregates calculated using future rows, and preprocessing performed before the train/test split.

A data contract records expected column names, types, allowed ranges, missing-value rules, and the target definition. Assertions such as assert not X.isna().any().any() turn quiet corruption into a visible failure.

Scenario: A loan model uses collection_outcome as a feature. Offline accuracy is 99%, but the field is filled weeks after approval. Removing it lowers the score and makes the evaluation honest.

Goal: Produce exactly the safe numeric features requested by the lab while preserving row alignment with the target.