Questions tagged [alphazero]

For questions related to DeepMind's AlphaZero, which is a computer program that can play Go, Chess, and Shogi. AlphaZero achieved, within 24 hours of training, a superhuman level of play in these three games by defeating world-champion programs Stockfish, Elmo, and the 3-day version of AlphaGo Zero. AlphaZero was introduced in "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm" (2017) by David Silver et al.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

AlphaZero value at root node not being affected by training

I have written my own AlphaZero implementation and started training it recently. Problem is, I am 99% sure there is a mistake and I do not know how to tackle this, since I cannot explain it. I am new ...
1
vote
1answer
45 views

When does AlphaZero play suboptimal moves?

If AlphaZero was always playing the best moves it would just generate the same training game over and over again. So where does the randomness come from? When does it decide not to play the most ...
1
vote
0answers
24 views

Alphazero Value loss doesn't decrease

Currenly I'm trying to reimplement alphazero in pure c++ using libtorch to accomodate my project's need. But when I training my model, I found out that the value loss doesn't decrese at all after even ...
2
votes
1answer
47 views

What does it mean for AlphaZero's network to be “fully trained”

Reading this blog post about AlphaZero: https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go It uses language such as "the amount of training the network ...
-1
votes
1answer
56 views

What's missing from Alpha Zero to make it generally intelligent?

Alpha Zero is the game playing AI. One might try to use the algorithm in a robot which takes input from it's environment and thinks about taking the best course of action. So I was thinking about ...
1
vote
1answer
27 views

Alpha Zero queen promotion

"The final 9 planes encode possible underpromotions for pawn moves or captures in two possible diagonals, to knight, bishop or rook respectively. Other pawn moves or captures from the ...
4
votes
1answer
43 views

How can I use one neural network for both players in Alpha Zero (Connect 4)?

First of all, it is great to have found this community! I am currently implementing my own Alpha Zero clone on Connect4. However, I have a mental barrier I cannot overcome. How can I use one neural ...
6
votes
1answer
151 views

Does AlphaZero use Q-Learning?

I was reading the AlphaZero paper Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm, and it seems they don't mention Q-Learning anywhere. So does AZ use Q-...
3
votes
1answer
65 views

Knowledge required for understanding AlphaZero paper

My goal is to understand AlphaZero paper published by deepmind. I'm beginning my journey trying to get the basic intuition of reinforcement learning from the book by Barto and Sutton. As per my ...
1
vote
1answer
37 views

How to deal with invalid output in a policy network? [duplicate]

I am interested in creating a neural network-based engine for chess. It uses a $8 \times 8 \times 73$ output space for each possible move as proposed in the Alpha Zero paper: Mastering Chess and Shogi ...
0
votes
1answer
114 views

How does AlphaZero use its value and policy heads in conjunction?

I have a question about how the value and policy heads are used in AlphaZero (not Alphago Zero), and where the leaf nodes are relative to the root node. Specifically, there seem to be several possible ...
2
votes
1answer
102 views

Alphazero policy head loss not decreasing

I am now working on training an alphazero player for a board game. The implementation of board game is mine, MCTS for alphazero was taken elsewhere. Due to complexity of the game, it takes a much ...
4
votes
2answers
71 views

How can alpha zero learn if the tree search stops and restarts before finishing a game?

I am trying to understand how alpha zero works, but there is one point that I have problems understanding, even after reading several different explanations. As I understand it (see for example https:/...
1
vote
2answers
75 views

Combining deep reinforcement learning with alpha-beta pruning

I will explain my question in relation to chess, but it should be relevant for other games as well: In short terms: Is it possible to combine the techniques used by AlphaZero with those used by, say, ...
1
vote
1answer
51 views

Similarities and differences between UCT algorithms in (i), (ii), (iii) and (iv)?

I am trying to understand the similarities and differences between: (i) the UCT algorithm in Kocsis and Szepesvári (2006); (ii) the UCT algorithm in Section 3.3 of Browne et al (2012); (iii) the MCTS ...
5
votes
2answers
298 views

What is the difference between DQN and AlphaGo Zero?

I have already implemented a relatively simple DQN on Pacman. Now I would like to clearly understand the difference between a DQN and the techniques used by AlphaGo zero/AlphaZero and I couldn't find ...
3
votes
1answer
94 views

Alpha zero before move 8

The Alpha zero paper says that the The first set of features are repeated for each position in a T = 8-step history. So what happens before the first 8 moves? Do they just repeat the starting position?...
5
votes
1answer
764 views

Why does the policy network in AlphaZero work?

In AlphaZero, the policy network (or head of the network) maps game states to a distribution of the likelihood of taking each action. This distribution covers all possible actions from that state. ...
4
votes
2answers
376 views

AlphaZero Value Network

The Alpha Zero (as well as AlphaGo Zero) papers say they trained the value head of the network by "minimizing the error between the predicted winner and the game winner" throughout its many self play ...
6
votes
3answers
176 views

Would AlphaGo Zero become perfect with enough training time?

Would AlphaGo Zero become theoretically perfect with enough training time? If not, what would be the limiting factor? (By perfect, I mean it always wins the game if possible, even against another ...
8
votes
2answers
414 views

Does Monte Carlo Search (specifically used by AlphaZero) Qualify as Machine Learning?

To the best of my understanding, Monte Carlo Search is an alternative method to Minimax for searching a tree of nodes. It works by choosing a move (generally the one with the highest chance of being ...
-4
votes
1answer
153 views

Are recent advances in machine learning really “artificial” intelligence, or merely brute force and human design?

It sounds like people boast of something being "artificial" about machine learning when actually people boast that humans implemented algorithms like e.g. Monte Carlo Search (MCST) etc. I think the ...
9
votes
3answers
790 views

Why were Chess experts surprised by the AlphaZero's victory against Stockfish?

It was recently brought to my attention that Chess experts took the outcome of this now famous match as something of an upset. See: Chess’s New Best Player Is A Fearless, Swashbuckling Algorithm ...
1
vote
1answer
356 views

What was the average decision speed pf Alpha Zero in the recent Stockfish match?

The match got a lot of press, and I doubt anyone is surprised that Alpha Zero crushed Stockfish. See: AlphaZero Destroys Stockfish in 100 Game Match To me, what's really salient is that "much like ...