Importance Is Not Explanation

Forests trade clarity for robustness

Random forests are strong general-purpose tabular models, robust to nonlinearities and feature interactions. They can be slower and larger than linear models, do not extrapolate regression targets naturally, and lose the single-tree flowchart explanation.

Built-in impurity-based feature importance measures how much a feature reduced impurity across splits, but it can favor continuous or high-cardinality features. Permutation importance shuffles one feature in validation data and measures the performance drop, giving a more direct dependence test. Correlated features can share or mask importance in either method.

from sklearn.inspection import permutation_importance
imp = permutation_importance(forest, X_valid, y_valid, n_repeats=10, random_state=42)

Warning: Importance says the model used a feature for prediction. It does not say the feature causes the outcome, is ethically appropriate, or should be manipulated.

Tip: Reach for a forest as a dependable nonlinear baseline on tabular data, particularly when preprocessing time is limited and single-tree instability is unacceptable.