Detect and Prevent Leakage

Suspiciously good deserves investigation

Leakage signals include near-perfect metrics on a difficult problem, a dramatic gap between offline and production performance, one feature dominating importance, or scores collapsing under a time/group split. These clues do not prove leakage, but they justify tracing how every high-impact feature was created.

Prevention is structural: define a prediction timestamp, build features using only records at or before it, split by the deployment boundary, fit all transforms inside a pipeline, deduplicate before splitting, and review joins for one-to-many surprises. An ablation test retrains after removing a suspicious feature; an implausibly large score drop is evidence to inspect its provenance.

full features:             F1 = 0.98
without post_event_status: F1 = 0.71
action: trace post_event_status; do not celebrate 0.98

Note: Leakage is a process failure more often than an algorithm failure. A simpler model with honest features is more valuable than a sophisticated model evaluated with contaminated evidence.

Goal: The ml-data-hygiene challenge includes a target-leaking column. Remove it while preserving row-to-label alignment and prove the resulting feature contract is honest.