Nearest neighbours and naive Bayes
Two of the oldest classifiers make opposite bets. k-nearest neighbours assumes almost nothing and remembers everything; naive Bayes assumes something false and still works. Learn both, and meet the curse of dimensionality along the way.
40 min
After this lesson you can
- Classify with k-nearest neighbours, and explain the choice of as a bias–variance trade-off
- Explain why feature scaling and the distance metric change a k-NN classifier
- Show numerically that distances concentrate in high dimensions, and say what that does to "nearest"
- Derive the naive Bayes classifier from Bayes' rule plus conditional independence, and fit Gaussian and multinomial versions by counting
- Explain why naive Bayes classifies well even when its assumption is false, and why its probabilities should not be trusted
In this lesson
- Classify by remembering
- What “near” means
- The curse of dimensionality
- Naive Bayes: assume independence, then count
- In code
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.