4
votes
Would AlphaZero perform better if made with transformers?
Yes!
LeelaChessZero, an open-source re-implementation and continuation of AlphaZero, has been experimenting with this for a while now. Their strongest networks are currently transformers, not ...
4
votes
Accepted
Deep Q Networks v Monte Carlo Tree Search in Alpha Zero
DQN and MCTS (+ NNs) are algorithms of two distinct RL flavors:
DQN is model-free RL: it's able to learn the value of actions without having access to the underlying MDP (i.e. environment's model), ...
2
votes
Accepted
Reproducing AlphaZero/MuZero: Failed to beat initial model in arena
I've tried to organize my answer as best as possible, first answering your specific questions and then giving some higher level thoughts.
Using score as the value
PUCT as used in AlphaZero doesn't ...
2
votes
Accepted
Why does training a NN using MCTS work even if the number of simulations isn't much larger than the number of actions?
Your understanding of AlphaZero-style MCTS seems correct, except that tree traversal does not happen randomly! It uses the PUCT formula to pick the child node. Similarly the policy is not simply the ...
1
vote
Accepted
AlphaZero Chess: High portion of "draws" during first rounds of self-play hampers learning
First intuition
I wrote a quick script to test the outcome distribution of random chess games: I get about 7% wins for black and white each, so 14% non-drawn games. The MCTS should still be able to ...
1
vote
Does AlphaGo play random moves in a real competition?
AlphaGo/AlphaZero has 3 main sources of randomness during competitive mode:
Move temperature: The MCTS process outputs a probability distribution P over all candidate moves. The agent chooses a move ...
1
vote
How does AlphaZero's MCTS work when starting from the root node?
I had the same question and found this page, then I also started to look at the pseudo-code of Supplementary Materials. The code of the computation of the UCB score is as followed:
...
Only top scored, non community-wiki answers of a minimum length are eligible
Related Tags
alphazero × 74reinforcement-learning × 40
monte-carlo-tree-search × 27
chess × 17
deep-rl × 13
neural-networks × 12
alphago-zero × 12
alphago × 11
deep-learning × 9
game-ai × 8
muzero × 5
machine-learning × 4
training × 4
deepmind × 4
ai-design × 3
policy-gradients × 3
multi-agent-systems × 3
reference-request × 2
papers × 2
dqn × 2
agi × 2
game-theory × 2
alpha-beta-pruning × 2
explainable-ai × 2
self-play × 2