Bias Enters Through the Pipeline

Models inherit selection and measurement choices

Bias in ML is a systematic error that affects some outcomes or groups differently. It can enter through who gets sampled, which behavior receives a label, how features are measured, which objective is optimized, and how predictions drive future data collection. A model can accurately reproduce historical decisions while perpetuating historical inequity.

Evaluate important metrics by meaningful slices - subgroups such as region, device type, language, or accessibility mode - while respecting privacy and avoiding tiny groups that make estimates unstable or identifying. Similar overall accuracy can hide one slice with much higher false-negative rates.

overall recall: 0.84
new devices:    0.91
older devices:  0.52  <- investigate representation and sensor behavior

Warning: Removing a protected attribute does not automatically remove its influence. Postal code, device, employer, or browsing behavior may act as a proxy feature, indirectly carrying much of the same information.

Note: Fairness has multiple definitions that can conflict. Choose measures based on affected people, real harms, law, and product context - not whichever dashboard number is easiest to optimize.