Fit on Training, Transform Everywhere

PCA learns directions from data

PCA is a fitted transformation: it learns feature means and component directions. Standardize features first when their units differ, then fit both scaler and PCA on training data only. Apply the stored transforms unchanged to validation, test, and production inputs.

from sklearn.decomposition import PCA
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

pipe = make_pipeline(
    StandardScaler(),
    PCA(n_components=0.95),
    classifier,
)
pipe.fit(X_train, y_train)

n_components=0.95 keeps enough components to explain 95% of training variance. The explained variance ratio reports how much variance each component retains. Cross-validation should still decide whether compression improves the actual downstream metric.

Scenario: A team fits PCA on the full dataset before splitting. Test rows influence the learned directions, creating preprocessing leakage even though target labels were never used.

Tip: Keep PCA inside the pipeline and version original feature order. Components cannot be reconstructed correctly if columns arrive reordered.