Why Fewer Dimensions Can Help

Remove redundancy, not information blindly

Dimensionality reduction represents data with fewer numeric features while preserving useful structure. It can reduce noise, speed training, visualize high-dimensional data, and help distance-based algorithms. It is not automatically beneficial: transformed features may be harder to explain and predictive information can be discarded.

Principal Component Analysis (PCA) finds new perpendicular directions called principal components. The first captures the greatest variance in the data, the second captures the greatest remaining variance while perpendicular to the first, and so on. Components are weighted combinations of original features.

original: CPU, memory, disk, network (4 axes)
PCA: component 1 = broad load pattern
     component 2 = compute-heavy versus I/O-heavy pattern

Analogy: A three-dimensional object casts many two-dimensional shadows. PCA chooses the viewing angle whose shadow preserves the most visible spread, but depth information is inevitably lost.

Warning: High variance is not the same as target relevance. A low-variance feature may be the clearest signal for a rare class.