Distance Depends on Scale

Units shape the neighborhood

KNN commonly uses Euclidean distance, the straight-line distance across feature coordinates. If income ranges from 0 to 100,000 while ticket count ranges from 0 to 10, income overwhelms the distance even when ticket count is more relevant. Standardization puts numeric features on comparable scales.

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
pipe = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=7))
pipe.fit(X_train, y_train)

Feature selection matters because irrelevant dimensions add noise to distance. The chosen metric also encodes meaning: straight-line distance, Manhattan step distance, or a domain-specific measure can produce different neighbors.

Scenario: A model classifies servers using CPU percentage and disk bytes. Without scaling, a one-gigabyte disk difference dominates a fifty-point CPU difference purely because bytes use larger numbers.

Warning: Fit the scaler on training data inside a pipeline. Scaling the full dataset before splitting leaks held-out statistics.