The multilayer perceptron
Neurons, activation functions, and layers that warp space until the classes come apart — the forward pass in matrix form, and exactly what the universal approximation theorem promises.
40 min
After this lesson you can
- Write a neuron and a layer as equations, and track the shape of every matrix in a forward pass
- Explain why a nonlinearity is essential by showing that stacked linear layers collapse into one
- See a trained network as a sequence of warps of space that makes the classes linearly separable
- State the universal approximation theorem precisely, and list three things it does not promise
In this lesson
- A neuron, and why the nonlinearity matters
- A layer, and a stack of layers
- Layers warp space
- A playground
- What a network can represent: universal approximation
- How the frameworks write it
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.