Why Deep Trees Memorize

Unlimited questions isolate individual rows

An unconstrained tree can keep splitting until leaves contain one or a few training examples. Training accuracy approaches perfection, but tiny changes in data may produce a very different structure. This is high variance and classic overfitting.

Control complexity with max_depth, min_samples_split, min_samples_leaf, or cost-complexity pruning, which removes branches whose small training improvement does not justify added complexity. Choose these settings using cross-validation, not the final test set.

tree = DecisionTreeClassifier(
    max_depth=5, min_samples_leaf=20, random_state=42
)

Scenario: A leaf says if account_age=47 days and region=west and browser=old version, predict fraud based on one training row. That is memorization disguised as a rule. Requiring larger leaves makes patterns earn support from multiple examples.

Warning: A readable tree can still encode unstable or discriminatory rules. Interpretability makes inspection possible; it does not make the learned policy automatically trustworthy.