Random Forests Add Feature Randomness

Make trees disagree usefully

A random forest combines bagging with a random subset of candidate features at each split. Without feature randomness, one dominant feature may make every tree nearly identical; correlated trees make the same errors, reducing the benefit of averaging. Feature subsampling creates useful diversity.

from sklearn.ensemble import RandomForestClassifier
forest = RandomForestClassifier(
    n_estimators=300, max_features='sqrt',
    min_samples_leaf=5, random_state=42, n_jobs=-1,
)
forest.fit(X_train, y_train)

n_estimators is the number of trees. More trees generally stabilize estimates but cost CPU and memory. Depth and leaf size still control how complex individual trees become. Random forests need little scaling and capture nonlinear interactions well.

Scenario: One decision tree's recall swings ten points when the split seed changes. A 300-tree forest produces much steadier results because no single bootstrap sample controls the decision.

Tip: Plot validation performance and inference cost as tree count grows. Stop when added trees no longer provide meaningful stability or quality.