The limits of linear
Every linear classifier draws a straight boundary, and four points are enough to defeat it. Hand-made features and kernels can bend the boundary — but only a model that learns its own features escapes the trap for good.
30 min
After this lesson you can
- Prove that XOR is not linearly separable, and explain why no training procedure for a linear model can fix that
- Build a feature map by hand that makes XOR separable, and see the resulting curved boundary
- Count how quickly polynomial features grow, and explain what the kernel trick does and does not buy you
- State the question a hidden layer answers — can the features themselves be learned?
In this lesson
- Why every linear classifier draws a straight line
- Escape route 1: invent better features
- Escape route 2: the kernel trick (in one section)
- Escape route 3: learn the features
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.