3
$\begingroup$

From the AlphaGo Zero paper, AlphaGo Zero uses an exponentiated visit count from the tree search.

Why use visit count instead of the mean action value $Q(s, a)$?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The answer is surprisingly hidden in the original AlphaGo paper:

At the end of search AlphaGo selects the action with maximum visit count; this is less sensitive to outliers than maximizing action value.

Unfortunately, there did not appear to be further details in the paper or in the related reference. The root child node (corresponding to an action) with maximum visit count is fittingly known as the robust child, as described here and referenced in a MCTS survey here.

Improve this answer
$\endgroup$
1
  • $\begingroup$ One obvious scenario where mean actual value falls short is when the count is small (say, 1). $\endgroup$
    – dshin
    Commented Oct 25, 2022 at 20:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .