Descent
X.1Why Neural NetworksPro

A short history of the neuron

From a threshold unit in 1943 to AlexNet in 2012 — two collapses, one missing algorithm, and why "learn it from data at scale" kept winning.

35 min

After this lesson you can

  • Place the landmark ideas (threshold unit, perceptron, LMS, backprop, deep nets) on a timeline and say what problem each one solved
  • State precisely what Minsky and Papert proved in 1969, and separate it from how the book was received
  • Explain why training hidden units was the missing piece, and who contributed reverse-mode differentiation and backpropagation
  • Run the perceptron learning rule and watch it succeed on AND and fail forever on XOR

In this lesson

  1. A unit that fires
  2. The perceptron: weights that learn
  3. Eighty years on one axis
  4. 1969: what Perceptrons proved
  5. The missing piece: training the hidden units
  6. The second wave, and its ebb
  7. The third wave: depth, data, and GPUs
  8. A lens: the bitter lesson
  9. In code: the perceptron, succeeding and failing

This lesson is part of Descent Pro

Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.

See plansSign in / Sign up

Descent Pro

Start your free trial to keep reading

This lesson, and every other lesson on the route, is part of Descent Pro. Every plan starts with a free trial, and students with a .edu email get longer. Cancel before the trial ends and you pay nothing.

Start free trialSign in / Sign up