Decision trees
Ask one yes-or-no question about one feature, split the data, and repeat. Build a tree greedily with Gini impurity or information gain, see why it overfits and how pruning helps, and meet the instability that the next lesson turns into a strength.
40 min
After this lesson you can
- Describe a decision tree as a recursive, axis-aligned partition of the input space with a constant prediction in each region
- Compute Gini impurity, entropy and information gain for a candidate split, and explain why they are preferred to the error rate for growing trees
- Explain why tree growing is greedy, and give an example (XOR) where greed fails
- Control tree size by stopping rules and by cost-complexity pruning, and relate depth to bias and variance
- Explain why trees are interpretable, scale-free and unstable
In this lesson
- A tree is a partition
- Growing it: one greedy question at a time
- Measuring impurity
- Where greed goes wrong
- How big should the tree be?
- Regression trees
- What trees are good and bad at
- In code
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.