How Does AlphaGo Zero Implement Reinforcement Learning?

Question

AlphaGo Zero (https://deepmind.com/blog/alphago-zero-learning-scratch/) has several key components that contribute to it's success:

1. A Monte Carlo Tree Search Algorithm that allows it to better search and learn from the state space of Go
2. A Deep Neural Network architecture that learns the value and policies of given states, to better inform the MCTS.

My question is, how is this Reinforcement Learning? Or rather, what aspects of this algorithm specifically make it a Reinforcement Learning problem? Couldn't this just be considered a Supervised Learning problem?

Answer 1

If you learn a policy or a value function from experience (that is, interaction with an environment), that's RL. In the case of AlphaGo, the MCTS is used to acquire the experience.

RL could in fact be considered supervised learning (SL) or, more specifically, self-supervised learning, where the experience corresponds to the labels in SL, especially nowadays with techniques like experience replay.