2
$\begingroup$

From the AlphaGo Zero paper, AlphaGo Zero uses an exponentiated visit count from the tree search.

Why use visit count instead of the mean action value $Q(s, a)$?

$\endgroup$
2
$\begingroup$

The answer is surprisingly hidden in the original AlphaGo paper:

At the end of search AlphaGo selects the action with maximum visit count; this is less sensitive to outliers than maximizing action value.

Unfortunately, there did not appear to be further details in the paper or in the related reference. The root child node (corresponding to an action) with maximum visit count is fittingly known as the robust child, as described here and referenced in a MCTS survey here.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.