# Tag Info

Accepted

### Why does the policy network in AlphaZero work?

The output of the policy network is as described in the original paper: A move in chess may be described in two parts: selecting the piece to move, and then selecting among the legal moves for ...
• 23.1k
Accepted

### Why were Chess experts surprised by the AlphaZero's victory against Stockfish?

Good question. First and foremost is that in Go deepmind had no superhuman opponents to challenge. Go engines were not anywhere near the highest level of the top human players. In chess, however, ...
• 176
Accepted

### Is AlphaZero an example of an AGI?

Good question! AlphaZero, though a major milestone, is most definitely not an AGI :) AlphaGo, though strong at the game of Go, is narrowly strong ("strong-narrow AI"), defined as strength in a ...
• 6,077

### Does Monte Carlo tree search qualify as machine learning?

John's answer is correct in that MCTS is traditionally not viewed as a Machine Learning approach, but as a tree search algorithm, and that AlphaZero combines this with Machine Learning techniques (...
• 9,316
Accepted

### Does Monte Carlo tree search qualify as machine learning?

Monte Carlo Tree Search is not usually thought of as a machine learning technique, but as a search technique. There are parallels (MCTS does try to learn general patterns from data, in a sense, but ...
• 8,865
Accepted

### Does AlphaZero use Q-Learning?

Note: you mentioned in the comments that you are reading the old, pre-print version of the paper describing AlphaZero on arXiv. My answer will be for the "official", peer-reviewed, more recent ...
• 9,316
Accepted

### What is the difference between DQN and AlphaGo Zero?

DQN and AlphaZero do not share much in terms of implementation. However, they are based on the same Reinforcement Learning (RL) theoretical framework. If you understand terms like MDP, reward, return, ...
• 23.1k
Accepted

### Clarifying representation of Neural Nerwork input for Chess Alpha Zero

For anyone wondering, I believe to have found the answer: Yes, it will be an 8x8 plane where all the entries are the same, the number of moves (or mpves with no progress). There are two repetitions ...

### Why were Chess experts surprised by the AlphaZero's victory against Stockfish?

MCTS for chess had been tried in the literature with little success. It was assumed AlphaGo's approach would never work on chess, maybe in Go but not in chess. Suddenly, Google announced the approach ...
• 1,368

### Would AlphaGo Zero become perfect with enough training time?

Assuming you mean a mathematically perfect player, similar to what we can achieve trivially in Tic Tac Toe, then the answer is "maybe". The underlying reinforcement learning algorithms that it uses do ...
• 23.1k
Accepted

### What happens before the first 8 moves in Alpha Zero?

On page 13, right under Table S1 in the linked paper, this is explained (emphasis in bold at the end mine): Each set of planes represents the board position at a time-step $t - T + 1, \dots, t$, ...
• 9,316
Accepted

### What knowledge is required for understanding the AlphaZero paper?

The more you read, the more deeply you can understand any paper, but given your stated background, reading the Monte-Carlo Tree Search chapter of Barto & Sutton, plus Gerald Tesauro's TD-Gammon ...
• 8,865
Accepted

### How does the Alpha Zero's move encoding work?

Let's do the code, so all the details are down. Encoding dictionary: ...
• 1,763
Accepted

### What does it mean for AlphaZero's network to be "fully trained"

Neural network will eventually reach limit of it's approximation power. You can't expect to learn more and more things infinitely long with the same amount of learnable parameters. Also, if you ...
• 2,216

### Alpha Zero queen promotion

It means that there is no explicit coding of action choices to promote to queen, it is the default assumption if the underpromotion actions are not taken. The Alpha Zero chess implementation can ...
• 23.1k
Accepted

### How can I use one neural network for both players in Alpha Zero (Connect 4)?

Let's define your problem from another point of view. Let's say that in this RL problem you have two agents (agent1 and agent2) that compete with each other in order to accomplish their own goal, i.e.,...
• 46
Accepted

### Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?

Depends on perspective. On one hand, you have an agent playing in an environment with another agent also evolving. This falls under the definition of Multi-Agent Learning, as can be seen with works ...
• 856
Accepted

### How does AlphaZero use its value and policy heads in conjunction?

I am also a bit confused by your wording but I will try to clear some things up. During MCTS the policy head is used to guide the search while the value head is used as a replacement for roll outs to ...
• 46
Accepted

### Is there any value given to each chess piece in AlphaZero?

is there a value given for each piece (e.g. 1 for pawn, 3 for knight, 9 for queen, etc.) to train the algorithm, or does the algorithm learn this by himself? No, there are no such explicit values ...
• 9,316
Accepted

### What part of the game is the value network trained to predict a winner on?

To my understanding, this is basically a supervised learning problem, where from the self play we have games associated with their winners, and the network is being trained to map game states to ...
• 23.1k
Accepted

### Would AlphaGo Zero become perfect with enough training time?

We cannot tell with certainty whether AlphaGo Zero would become perfect with enough training time. This is because none of the parts (Neural Network) that would benefit from infinite training time (= ...
• 9,316
Accepted

### Is it practical to train AlphaZero or MuZero (for indie games) on a personal computer?

The vast majority of neural networks are now trained on graphics processing units (GPUs) or specialised accelerator hardware such as tensor processing units (TPUs). In Mastering Chess and Shogi by ...
• 931
Accepted

### What is a policy training target in AlphaZero?

The training target for the policy is always exactly the one described by the pseudocode; distribution proportional to visit counts, without any other kind of scaling. The softmax sampling with the ...
• 9,316

### Why were Chess experts surprised by the AlphaZero's victory against Stockfish?

I see, based on the articles you provide, many levels of surprise in the victory: Chess is hard game to master and the counter part had the world's best practices, AlphaZero had tabula rasa. ...
• 927
Accepted

### Alphazero policy head loss not decreasing

The loss of the policy head here is really quite different from losses in, for instance, more "conventional" Supervised Learning approaches (where we typically expect/hope to see a relatively steady ...
• 9,316
Accepted

### How can alpha zero learn if the tree search stops and restarts before finishing a game?

You're right that AlphaGo Zero doesn't perform rollouts in its MCTS. It does complete many, many games, though. Realize that AlphaGo Zero only iterates MCTS 1,600 times before taking an action. The ...
• 1,918

### Combining deep reinforcement learning with alpha-beta pruning

Yes it's possible to to combine AlphaZero with Minimax methods (including alpha-beta pruning). AlphaZero itself is combination of Monte Carlo Tree Search (MCTS) and Deep Network, where MCTS is used ...
• 655
Accepted

### When does AlphaZero play suboptimal moves?

During the self-play training process, AlphaZero does not greedily play only the moves it thinks are "best" (which would normally be the move with the highest visit count leading out of the root node ...
• 9,316