What does it mean there is no rollout in AlphaZero's training?

Question

According to a blog post by DeepMind, AlphaZero doesn't have a real rollout.

AlphaGo Zero does not use "rollouts" - fast, random games used by other Go programs to predict which player will win from the current board position. Instead, it relies on its high quality neural networks to evaluate positions.

Instead, I assume it just interprets the winner at a given state by the NN values head result. This replaces the rollout. So the computation time saved could be used for many expansions instead. Evaluating a state from a root node would then be the best action derived from the visit count in MCTS, which is only based on the predictions of the NN value heads. (no current score, no policy?)

With policy, I mean the NN's policy head (softmax).

This would mean that the NN policy is only used in the loss calculation and nowhere else?

Answer 1

TL;DR: Alpha Zero removed rollout altogether from MCTS and just used current DNN estimates instead.

---

The single Deep Neural Network has 2 heads:

• A Value Head (which assigns a score to each state).
• And a Policy Head (which predicts the score for all possible moves).

Instead of doing rollouts to determine the outcome, it uses the DNN estimations, so it don't need to explore too deep.

By relying on the DNN, the MCTS gets even simpler:

• Probability of each action is a simple normalization of previously obtained Policy Head.
• Selection chooses move with “low count, high move probability, and high value”
• Expansion is done by DNN, outputing Value and Policy.
• Simulation with rollouts: no longer needed.
• Back-propagation updates nodes using DNN's value (instead of a rollout outcome).

Source