7
$\begingroup$

In the vanilla Monte Carlo tree search (MCTS) implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only then the information gathered is backed up.

I have read the AlphaZero paper (and the AlphaGo Zero too) and I didn't find any information on how the rollout is implemented (maybe I missed it).

How is the rollout from the MCTS implemented in both the AlphaGo Zero and the AlphaZero algorithms?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ AlphaZero/Go don't have rollout. Rollout replaced with neural network value estimation $\endgroup$
    – mirror2image
    Nov 3, 2019 at 7:40
  • $\begingroup$ @mi: I thought that original Alpha Go did have a rollout policy, but that one of the simplifying changes in AlphaZero was to completely remove it? $\endgroup$
    – Neil Slater
    Nov 3, 2019 at 9:03
  • 1
    $\begingroup$ @NeilSlater That's correct. The question seems to be about AlphaGo Zero and AlphaZero though, not about Alpha Go :) AlphaGo: uses a fast rollout policy (trained like the policy network, but is not a large DNN, just a single layer + softmax I believe). AlphaGo Zero: no rollouts. AlphaZero: no rollouts. $\endgroup$
    – Dennis Soemers ♦
    Nov 3, 2019 at 13:53
  • $\begingroup$ @mirror2image even though it was only a line, you answered the question -- why not make it slightly more verbose and make it an answer $\endgroup$
    – mshlis
    Nov 3, 2019 at 17:57
  • $\begingroup$ Thanks for the answers. However, I'm still confused. I understand that they use the V value returned from the network to simulate who won, but how do they know which value of V a player won or lost? Because as far as I understand, in the weight update (in the loss function), they will compare the probability distribution gathered from the self-plays and the difference between the real winner (from MCTS) and the value V. $\endgroup$
    – ihavenoidea
    Nov 3, 2019 at 23:18

0

Reset to default

You must log in to answer this question.