Reinforcement learning foundations
States, actions, rewards and returns; the Bellman equations that let a reward at the end tell the first step what it is worth; and the two classic ways to learn from them — dynamic programming and Q-learning.
50 min
After this lesson you can
- Write down an MDP and the discounted return, and say what each symbol means
- Derive the Bellman expectation equation from the definition of value
- Run value iteration by hand and explain why it converges
- Implement tabular Q-learning with ε-greedy exploration
- Explain what DQN added to make Q-learning work with neural networks
In this lesson
- The agent–environment loop
- Value functions: how good is it to be here?
- Dynamic programming: value iteration
- Learning without a model: temporal differences
- Function approximation and the deep-RL milestones
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.