Population, Sample, and Representation

The data you have stands in for data you will meet

The population is the complete set of cases the system is meant to handle. A sample is the finite subset actually collected. ML assumes patterns learned from the sample will generalize, meaning remain useful on new members of the population. That assumption weakens when collection systematically misses groups, time periods, devices, or rare conditions.

A sample can be large and still unrepresentative. Ten million desktop sessions do not describe mobile behavior if the product will serve both. Sampling bias occurs when the collection process makes some cases more likely to appear than others. Survivorship bias occurs when only cases that remained visible are analyzed, such as studying active customers while excluding everyone who already left.

Scenario: A failure detector trains only on incidents severe enough to receive a ticket. Quiet failures never entered the dataset, so the model learns the documentation habits of responders as much as the actual system behavior.

deployment population: all regions, all device versions, weekdays + weekends
training sample: one region, latest devices, weekday business hours only
result: excellent local test score, weak deployment coverage

Note: Representation is not fixed forever. Products, users, sensors, and policies change. Record how data was sampled and monitor whether production still resembles that intended population.