Multi-agent RL
When the environment contains other learners, "the best policy" stops being well defined — it depends on what everyone else does, and they are changing too. Games, Nash equilibria, why learners in competition cycle, how self-play and leagues tame it, how teams learn together, and what AlphaStar, OpenAI Five and hide-and-seek showed.
50 min
After this lesson you can
- Define a Markov game and a Nash equilibrium, and compute the equilibrium of a small matrix game
- Explain why other learners make the environment non-stationary
- Show that learners in a zero-sum game can cycle while their time-averages converge
- Describe self-play, fictitious play and AlphaStar's league, and centralised training with decentralised execution
- Say what the landmark systems achieved, under what restrictions, and what remains open
In this lesson
- From MDPs to Markov games
- The environment is learning back
- Self-play, from backgammon to StarCraft
- Learning as a team
- Emergence: hide-and-seek
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.