Questions tagged [alphago]
For questions related to DeepMind's AlphaGo, which is the first computer Go program to beat a human professional Go player without handicaps on a full-sized 19x19 board. AlphaGo was introduced in the paper "Mastering the game of Go with deep neural networks and tree search" (2016) by David Silver et al. There have been three more powerful successors of AlphaGo: AlphaGo Master, AlphaGo Zero and AlphaZero.
27
questions
1
vote
1
answer
63
views
When do we use the neural network to predict value during the expansion stage of MCTS in the AlphaZero algorithm?
According to what I understand from the AlphaZero algorithm, a neural network is used to set value and prior probability for a node during the expansion stage of MCTS. On the other hand, according to ...
1
vote
1
answer
99
views
Is it possible for AlphaGo Zero to use recurrent networks to achieve similar performance?
AlphaGo Zero stacks 7 board history along with the current board together to form the input to the network. However, is it possible to use an RNN to replace the input of history and achieve similar ...
- 213
2
votes
3
answers
390
views
Does AlphaGo play random moves in a real competition?
Alphago and AlphaGo zero use random play to generate data and use the data to train DNN. "Random play" means that there is a positive probability for AlphaGo to play some suboptimal moves ...
- 163
1
vote
0
answers
441
views
What would be the AlphaGo's performance in continuous action space?
During my research for Google DeepMind's Go-playing program Alpha Go and its successor Alpha Go Zero, I discovered that the system uses a clever pipeline and an interplay of blocks of both policy and ...
- 31
3
votes
1
answer
217
views
AlphaGo Zero: does $Q(s_t, a)$ dominate $U(s_t, a)$ in difficult game states?
AlphaGo Zero
AlphaGo Zero uses a Monte-Carlo Tree Search where the selection phase is governed by $\operatorname*{argmax}\limits_a\left( Q(s_t, a) + U(s_t, a) \right)$, where:
the exploitation ...
- 1,510
1
vote
1
answer
253
views
Initialising DQN with weights from imitation learning rather than policy gradient network
In AlphaGo, the authors initialised a policy gradient network with weights trained from imitation learning. I believe this gives it a very good starting policy for the policy gradient network. the ...
- 1,251
1
vote
1
answer
68
views
Is which sense was AlphaGo "just given a rule book"?
I was told that AlphaGo (or some related program) was not explicitly taught even the rules of Go -- if it was "just given the rulebook", what does this mean? Literally, a book written in ...
- 141
2
votes
1
answer
552
views
Did Alphago zero actually beat Alphago 100 games to 0?
tl;dr
Did AlphaGo and AlphaGo play 100 repetitions of the same sequence of boards, or were there 100 different games?
Background:
Alphago was the first superhuman go player, but it had human tuning ...
- 371
0
votes
1
answer
349
views
What is the search depth of AlphaGo and AlphaGo Zero?
I cannot find reliable sources but someone says it is 40 moves and someone else says it is 50+ moves. I read their papers and they use value function (NN) and policy function to trim the tree, so more ...
- 163
3
votes
0
answers
70
views
What does "convolve k filters" mean in the AlphaGo paper?
On page 27 of the DeepMind AlphaGo paper appears the following sentence:
The first hidden layer zero pads the input into a $23 \times 23$ image, then convolves $k$ filters of kernel size $5 \times 5$ ...
1
vote
1
answer
100
views
How AlphaGo Zero is learning from $\pi_t$ when $z_t = -1$?
I have questions on the way AlphaGo Zero is trained.
From original AlphaGo Zero paper, I knew that AlphaGo Zero agent learns a policy, value functions by the gathered data $\{(s_t, \pi_t, z_t)\}$ ...
- 13
3
votes
2
answers
214
views
How does the AlphaGo Zero policy decide what move to execute?
I was going through the AlphaGo Zero paper and I was trying to understand everything, but I just can't figure out this one formula:
$$
\pi(a \mid s_0) = \frac{N(s_0, a)^{\frac{1}{\tau}}}{\sum_b N(s_0,...
- 33
4
votes
1
answer
932
views
Alphazero policy head loss not decreasing
I am now working on training an alphazero player for a board game. The implementation of board game is mine, MCTS for alphazero was taken elsewhere. Due to complexity of the game, it takes a much ...
- 365
3
votes
1
answer
253
views
Why is Monte Carlo used as the tree search algorithm for AlphaGo?
Could a better algorithm other than Monte Carlo be used for the AlphaGo computer? Why didn't the DeepMind team think of choosing another kind of algorithm rather than spending time on their neural ...
- 343
2
votes
1
answer
232
views
Similarities and differences between UCT algorithms in (i), (ii), (iii) and (iv)?
I am trying to understand the similarities and differences between: (i) the UCT algorithm in Kocsis and Szepesvári (2006); (ii) the UCT algorithm in Section 3.3 of Browne et al (2012); (iii) the MCTS ...
5
votes
1
answer
116
views
Why didn't champion of the Go game manage to win the last game against AlphaGo, after winning the 4th one?
In the documentary about the match, it is said that after losing the 4th game, AlphaGo came back stronger and started to play in a weird way (not human-like) and it was pretty impossible to be beaten. ...
- 343
6
votes
1
answer
234
views
Why is a constant plane of ones added into the input features of AlphaGo?
In the paper Mastering the game of Go with deep neural networks and tree search, the input features of the networks of AlphaGo contains a plane of constant ones and a plane of constant zeros, as ...
- 61
10
votes
1
answer
1k
views
Is AlphaZero an example of an AGI?
From DeepMind's research paper on arxiv.org:
In this paper, we apply a similar but fully generic algorithm, which
we call AlphaZero, to the games of chess and shogi as well as Go,
without any ...
- 413
5
votes
2
answers
673
views
What part of the game is the value network trained to predict a winner on?
The Alpha Zero (as well as AlphaGo Zero) papers say they trained the value head of the network by "minimizing the error between the predicted winner and the game winner" throughout its many self-play ...
- 2,235
7
votes
3
answers
331
views
Would AlphaGo Zero become perfect with enough training time?
Would AlphaGo Zero become theoretically perfect with enough training time? If not, what would be the limiting factor?
(By perfect, I mean it always wins the game if possible, even against another ...
- 261
2
votes
5
answers
260
views
Is it fair to compare AlphaGo with a Human player?
A human player plays limited games compared to a system that undergoes millions of iterations. Is it really fair to compare AlphaGo with the world #1 player when we know experience increases with the ...
- 29
4
votes
1
answer
311
views
Why is chess still a benchmark for Artificial Intelligence?
Even though modern chess playing programs have demonstrated themselves to be as strong (or stronger) than even the best human players for nearly 20 years now (1997 when IBM's Deep Blue defeated the ...
- 143
4
votes
1
answer
1k
views
Would it take 1700 years to run AlphaGo Zero in commodity hardware?
From this link, AlphaGo would take millennia to run in regular hardware.
They generated 29 million games for the final result, which means it's going to take me about 1700 years to replicate this.
...
- 906
5
votes
1
answer
1k
views
Is the new AlphaGo implementation using Generative Adversarial Networks?
I read through the publication Mastering the game of Go without Human Knowledge. It doesn't seem to use GANs, just a new form of search and reinforcement learning.
- 163
5
votes
1
answer
2k
views
Why did AlphaGo lose its Go game?
We can read on wiki page that in March 2016 AlphaGo AI lost its game (1 of 5) to Lee Sedol, a professional Go player. One article cite says:
AlphaGo lost a game and we as researchers want to ...
- 10.4k
17
votes
1
answer
3k
views
How does "Monte-Carlo search" work?
I have heard about this concept in a Reddit post about AlphaGo. I have tried to go through the paper and the article, but could not really make sense of the algorithm.
So, can someone give an easy-to-...
- 1,371
7
votes
4
answers
219
views
Does the recent advent of a Go playing computer represent Artificial Intelligence?
I read that in the spring of 2016 a computer Go program was finally able to beat a professional human for the first time.
Now that this milestone has been reached, does that represent a significant ...
- 2,493