Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [muzero]

For questions about the MuZero algorithm proposed in the paper "Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model" (2019).

Filter by
Sorted by
Tagged with
1 vote
1 answer
114 views

Batch wise Inference to speed up Muzero's MCTS

Context: I've implemented Muzero for the game Tic-tac-toe. Unfortunately, the self-play and training is very slow (like 10 hours until it plays quite well). I ran the python profiler to find the ...
Lynix's user avatar
  • 13
1 vote
0 answers
55 views

RL agent focusses too much on early rewards, even with no discounting

How can I guide my RL agent to solve tasks in the correct order? I'm trying to train an agent using reinforcement learning, similar to MuZero. The goal is to solve 4 tasks, A/B/C/D. Each task involves ...
Christopher's user avatar
2 votes
1 answer
105 views

Why does training a NN using MCTS work even if the number of simulations isn't much larger than the number of actions?

tl;dr If the visit rates of children generated by MCTS is biased because not enough samples were taken, why doesn't the network learn random behavior? My understanding of combining MCTS and NNs (e.g. ...
Christopher's user avatar
1 vote
1 answer
348 views

Reproducing AlphaZero/MuZero: Failed to beat initial model in arena

I am trying to reproduce AlphaZero's algorithm on the board game Carcassonne. Since I want to use the final game score differences (i.e. victory point of player 1 - victory point of player 2) as the ...
TommyX's user avatar
  • 13
2 votes
2 answers
308 views

How are NNs output setup for games that allow multiple actions each turn and have very large sets of possible actions?

I was looking at an AI coding challenge for a two player game on a 2D grid of variable size (from one game to the next). Here is a screen shot example of the playfield. Each player has multiple units ...
snowfrogdev's user avatar
2 votes
1 answer
326 views

In the MuZero paper, how does backprop in the MCTS account for the immediate reward from each edge?

On page 12 of this paper: Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, it describes how MCTS works for the MuZero algorithm. It states in equation 4 that during the 'backup' ...
Matrix001's user avatar
  • 123
1 vote
1 answer
106 views

Scrabble rack observation with MuZero

Currently I'm trying to implement Scrabble with MuZero. The $15 \times 15$ game board observation (as input) is of size $27 \times15 \times15$ (26 letters + 1 wildcard) with a value of 0 or 1. However ...
Thrusticy's user avatar
5 votes
2 answers
2k views

Is it practical to train AlphaZero or MuZero (for indie games) on a personal computer?

Is it practical/affordable to train an AlphaZero/MuZero engine using a residential gaming PC, or would it take thousands of years of training for the AI to learn enough to challenge humans? I'm having ...
Luke W's user avatar
  • 53
5 votes
1 answer
672 views

Do AlphaZero/MuZero learn faster in terms of number of games played than humans?

I don't know much about AI and am just curious. From what I read, AlphaZero/MuZero outperform any human chess player after a few hours of training. I have no idea how many chess games a very talented ...
220284's user avatar
  • 153
3 votes
1 answer
217 views

How is MuZero's second binary plane for chess defined?

From the MuZero paper (Appendix E, page 13): In chess, 8 planes are used to encode the action. The first one-hot plane encodes which position the piece was moved from. The next two planes encode ...
MuZeroFm's user avatar
1 vote
1 answer
82 views

Is which sense was AlphaGo "just given a rule book"?

I was told that AlphaGo (or some related program) was not explicitly taught even the rules of Go -- if it was "just given the rulebook", what does this mean? Literally, a book written in ...
releseabe's user avatar
  • 141
2 votes
1 answer
138 views

How to choose the first action in a Monte Carlo Tree Search?

I'm working on reimplementing the MuZero paper. In the description of the MCTS (page 12), they indicate that a new node with associated state $s$ is to be initialized with $Q(s,a) = 0$, $N(s,a) = 0$ ...
Ziofil's user avatar
  • 128
3 votes
1 answer
331 views

How does MuZero learn to play well for both sides of a two-player game?

I'm coding my own version of MuZero. However, I don't understand how it supposed to learn to play well for both players in a two-player game. Take Go for example. If I use a single MCTS to generate an ...
Ziofil's user avatar
  • 128