4 votes

Would AlphaZero perform better if made with transformers?

Yes! LeelaChessZero, an open-source re-implementation and continuation of AlphaZero, has been experimenting with this for a while now. Their strongest networks are currently transformers, not ...
KarelPeeters's user avatar
4 votes
Accepted

Deep Q Networks v Monte Carlo Tree Search in Alpha Zero

DQN and MCTS (+ NNs) are algorithms of two distinct RL flavors: DQN is model-free RL: it's able to learn the value of actions without having access to the underlying MDP (i.e. environment's model), ...
Luca Anzalone's user avatar
2 votes
Accepted

Reproducing AlphaZero/MuZero: Failed to beat initial model in arena

I've tried to organize my answer as best as possible, first answering your specific questions and then giving some higher level thoughts. Using score as the value PUCT as used in AlphaZero doesn't ...
KarelPeeters's user avatar
2 votes
Accepted

Why does training a NN using MCTS work even if the number of simulations isn't much larger than the number of actions?

Your understanding of AlphaZero-style MCTS seems correct, except that tree traversal does not happen randomly! It uses the PUCT formula to pick the child node. Similarly the policy is not simply the ...
KarelPeeters's user avatar
1 vote
Accepted

AlphaZero Chess: High portion of "draws" during first rounds of self-play hampers learning

First intuition I wrote a quick script to test the outcome distribution of random chess games: I get about 7% wins for black and white each, so 14% non-drawn games. The MCTS should still be able to ...
KarelPeeters's user avatar
1 vote

Does AlphaGo play random moves in a real competition?

AlphaGo/AlphaZero has 3 main sources of randomness during competitive mode: Move temperature: The MCTS process outputs a probability distribution P over all candidate moves. The agent chooses a move ...
dshin's user avatar
  • 161
1 vote

How does AlphaZero's MCTS work when starting from the root node?

I had the same question and found this page, then I also started to look at the pseudo-code of Supplementary Materials. The code of the computation of the UCB score is as followed: ...
leodeng's user avatar
  • 11

Only top scored, non community-wiki answers of a minimum length are eligible