When One Tree Is the Right Tool

Clarity and nonlinear rules at modest scale

A shallow tree is valuable when decisions genuinely resemble thresholds, preprocessing should stay light, mixed feature relationships matter, and stakeholders need a flowchart-like explanation. Trees do not require standardization and can model nonlinearity directly.

Single trees are often less accurate and much less stable than ensembles. They also create axis-aligned rectangular regions, so a diagonal boundary may require many awkward steps. Regression trees predict a constant value inside each leaf and therefore cannot smoothly extrapolate beyond observed targets.

linear model: smooth additive boundary
single tree: explicit threshold rules
tree ensemble: many rules averaged, stronger but less transparent

Tip: Use a shallow tree when a reviewable decision aid matters; use a random forest or boosting when predictive performance matters more than reading one compact rule path.

Note: Scikit-learn trees require numeric encoded inputs and do not natively accept missing values in every estimator/version. Keep preprocessing explicit in the pipeline.