According to what I understand from the AlphaZero algorithm, a neural network is used to set value and prior probability for a node during the expansion stage of MCTS. On the other hand, according to the rules of the game, we need to set the value equal to +1 a win, -1 for a loss, or 0 for a draw.

Question: When do we use the neural network to predict the value and when do we use the rules of the game to set the value during the expansion stage of MCTS in AlphaZero?


1 Answer 1


The AlphaZero algorithm uses the neural network for non-terminal states and the game rules for terminal states when determining the value $v$ of a node during MCTS expansion. The supplementary material of the paper (arXiv link) states the following (see Domain Knowledge, item 2):

AlphaZero is provided with perfect knowledge of the game rules. These are used during MCTS, to simulate the positions resulting from a sequence of moves, to determine game termination, and to score any simulations that reach a terminal state.

During a game, AlphaZero stores the game outcome $z$ (determined by the game rules) and predicted outcomes $v_t$ of each encountered state (determined by the neural network). One aim of the loss function is to minimize the error between $z$ and $v_t$ (see equation 1 in the paper).


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .