Questions tagged [alphago]

For questions related to DeepMind's AlphaGo, which is the first computer Go program to beat a human professional Go player without handicaps on a full-sized 19x19 board. AlphaGo was introduced in the paper "Mastering the game of Go with deep neural networks and tree search" (2016) by David Silver et al. There have been three more powerful successors of AlphaGo: AlphaGo Master, AlphaGo Zero and AlphaZero.

Filter by
Sorted by
Tagged with
2 votes
2 answers
105 views

Does AlphaGo play random moves in a real competition?

Alphago and AlphaGo zero use random play to generate data and use the data to train DNN. "Random play" means that there is a positive probability for AlphaGo to play some suboptimal moves ...
user avatar
  • 141
1 vote
0 answers
252 views

What would be the AlphaGo's performance in continuous action space?

During my research for Google DeepMind's Go-playing program Alpha Go and its successor Alpha Go Zero, I discovered that the system uses a clever pipeline and an interplay of blocks of both policy and ...
user avatar
  • 21
3 votes
1 answer
109 views

AlphaGo Zero: does $Q(s_t, a)$ dominate $U(s_t, a)$ in difficult game states?

AlphaGo Zero AlphaGo Zero uses a Monte-Carlo Tree Search where the selection phase is governed by $\operatorname*{argmax}\limits_a\left( Q(s_t, a) + U(s_t, a) \right)$, where: the exploitation ...
user avatar
  • 1,210
1 vote
1 answer
101 views

Initialising DQN with weights from imitation learning rather than policy gradient network

In AlphaGo, the authors initialised a policy gradient network with weights trained from imitation learning. I believe this gives it a very good starting policy for the policy gradient network. the ...
user avatar
  • 1,201
1 vote
1 answer
58 views

Is which sense was AlphaGo "just given a rule book"?

I was told that AlphaGo (or some related program) was not explicitly taught even the rules of Go -- if it was "just given the rulebook", what does this mean? Literally, a book written in ...
user avatar
  • 131
2 votes
1 answer
295 views

Did Alphago zero actually beat Alphago 100 games to 0?

tl;dr Did AlphaGo and AlphaGo play 100 repetitions of the same sequence of boards, or were there 100 different games? Background: Alphago was the first superhuman go player, but it had human tuning ...
user avatar
0 votes
1 answer
125 views

What is the search depth of AlphaGo and AlphaGo Zero?

I cannot find reliable sources but someone says it is 40 moves and someone else says it is 50+ moves. I read their papers and they use value function (NN) and policy function to trim the tree, so more ...
user avatar
  • 141
3 votes
0 answers
43 views

What does "convolve k filters" mean in the AlphaGo paper?

On page 27 of the DeepMind AlphaGo paper appears the following sentence: The first hidden layer zero pads the input into a $23 \times 23$ image, then convolves $k$ filters of kernel size $5 \times 5$ ...
user avatar
1 vote
1 answer
90 views

How AlphaGo Zero is learning from $\pi_t$ when $z_t = -1$?

I have questions on the way AlphaGo Zero is trained. From original AlphaGo Zero paper, I knew that AlphaGo Zero agent learns a policy, value functions by the gathered data $\{(s_t, \pi_t, z_t)\}$ ...
user avatar
3 votes
2 answers
142 views

How does the AlphaGo Zero policy decide what move to execute?

I was going through the AlphaGo Zero paper and I was trying to understand everything, but I just can't figure out this one formula: $$ \pi(a \mid s_0) = \frac{N(s_0, a)^{\frac{1}{\tau}}}{\sum_b N(s_0,...
user avatar
  • 33
3 votes
1 answer
619 views

Alphazero policy head loss not decreasing

I am now working on training an alphazero player for a board game. The implementation of board game is mine, MCTS for alphazero was taken elsewhere. Due to complexity of the game, it takes a much ...
user avatar
  • 325
3 votes
1 answer
129 views

Why is Monte Carlo used as the tree search algorithm for AlphaGo?

Could a better algorithm other than Monte Carlo be used for the AlphaGo computer? Why didn't the DeepMind team think of choosing another kind of algorithm rather than spending time on their neural ...
user avatar
2 votes
1 answer
135 views

Similarities and differences between UCT algorithms in (i), (ii), (iii) and (iv)?

I am trying to understand the similarities and differences between: (i) the UCT algorithm in Kocsis and Szepesvári (2006); (ii) the UCT algorithm in Section 3.3 of Browne et al (2012); (iii) the MCTS ...
user avatar
5 votes
1 answer
94 views

Why didn't champion of the Go game manage to win the last game against AlphaGo, after winning the 4th one?

In the documentary about the match, it is said that after losing the 4th game, AlphaGo came back stronger and started to play in a weird way (not human-like) and it was pretty impossible to be beaten. ...
user avatar
6 votes
1 answer
161 views

Why is a constant plane of ones added into the input features of AlphaGo?

In the paper Mastering the game of Go with deep neural networks and tree search, the input features of the networks of AlphaGo contains a plane of constant ones and a plane of constant zeros, as ...
user avatar
  • 61
10 votes
1 answer
1k views

Is AlphaZero an example of an AGI?

From DeepMind's research paper on arxiv.org: In this paper, we apply a similar but fully generic algorithm, which we call AlphaZero, to the games of chess and shogi as well as Go, without any ...
user avatar
4 votes
2 answers
597 views

What part of the game is the value network trained to predict a winner on?

The Alpha Zero (as well as AlphaGo Zero) papers say they trained the value head of the network by "minimizing the error between the predicted winner and the game winner" throughout its many self-play ...
user avatar
7 votes
3 answers
263 views

Would AlphaGo Zero become perfect with enough training time?

Would AlphaGo Zero become theoretically perfect with enough training time? If not, what would be the limiting factor? (By perfect, I mean it always wins the game if possible, even against another ...
user avatar
  • 261
2 votes
5 answers
225 views

Is it fair to compare AlphaGo with a Human player?

A human player plays limited games compared to a system that undergoes millions of iterations. Is it really fair to compare AlphaGo with the world #1 player when we know experience increases with the ...
user avatar
  • 29
4 votes
1 answer
246 views

Why is chess still a benchmark for Artificial Intelligence?

Even though modern chess playing programs have demonstrated themselves to be as strong (or stronger) than even the best human players for nearly 20 years now (1997 when IBM's Deep Blue defeated the ...
user avatar
  • 143
3 votes
1 answer
1k views

Would it take 1700 years to run AlphaGo Zero in commodity hardware?

From this link, AlphaGo would take millennia to run in regular hardware. They generated 29 million games for the final result, which means it's going to take me about 1700 years to replicate this. ...
user avatar
5 votes
1 answer
1k views

Is the new AlphaGo implementation using Generative Adversarial Networks?

I read through the publication Mastering the game of Go without Human Knowledge. It doesn't seem to use GANs, just a new form of search and reinforcement learning.
user avatar
  • 163
5 votes
1 answer
2k views

Why did AlphaGo lose its Go game?

We can read on wiki page that in March 2016 AlphaGo AI lost its game (1 of 5) to Lee Sedol, a professional Go player. One article cite says: AlphaGo lost a game and we as researchers want to ...
user avatar
  • 10k
16 votes
1 answer
3k views

How does "Monte-Carlo search" work?

I have heard about this concept in a Reddit post about AlphaGo. I have tried to go through the paper and the article, but could not really make sense of the algorithm. So, can someone give an easy-to-...
user avatar
  • 1,341
7 votes
4 answers
191 views

Does the recent advent of a Go playing computer represent Artificial Intelligence?

I read that in the spring of 2016 a computer Go program was finally able to beat a professional human for the first time. Now that this milestone has been reached, does that represent a significant ...
user avatar
  • 2,293