Questions tagged [alphago-zero]

For questions related to AlphaGo Zero, which is a version of DeepMind's Go software, AlphaGo, that does not use data from human games and it is stronger than AlphaGo. There is a generalized version of AlphaGo Zero called AlphaZero, which beat the 3-day version of AlphaGo Zero by winning 60 games to 40. AlphaGo Zero was introduced in the paper "Mastering the game of Go without human knowledge" (2017) by David Silver et al.

Filter by
Sorted by
Tagged with
1 vote
1 answer
45 views

Is it possible for AlphaGo Zero to use recurrent networks to achieve similar performance?

AlphaGo Zero stacks 7 board history along with the current board together to form the input to the network. However, is it possible to use an RNN to replace the input of history and achieve similar ...
3 votes
1 answer
174 views

How does policy network learn in AlphaZero?

I'm currently trying to understand how AlphaZero works. There is one thing with the training of the AlphaZero's policy head that confuses me. Basically, in AlphaGo Zero's paper (where the major part ...
5 votes
1 answer
810 views

What is a "logit probability"?

DeepMind's paper "Mastering the game of Go without human knowledge" states in its "Methods" section on its "Neural network architecture" that the output layer of AlphaGo Zero's policy head is "A fully ...
1 vote
2 answers
210 views

In Alpha(Go)Zero, why is the policy extracted from MCTS better than the network one?

I've read through the Alpha(Go)Zero paper and there is only one thing I don't understand. The paper on page 1 states: The MCTS search outputs probabilities π of playing each move. These search ...
2 votes
2 answers
142 views

Does AlphaGo play random moves in a real competition?

Alphago and AlphaGo zero use random play to generate data and use the data to train DNN. "Random play" means that there is a positive probability for AlphaGo to play some suboptimal moves ...
5 votes
0 answers
543 views

How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?

In the vanilla Monte Carlo tree search (MCTS) implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only ...
3 votes
1 answer
1k views

Would it take 1700 years to run AlphaGo Zero in commodity hardware?

From this link, AlphaGo would take millennia to run in regular hardware. They generated 29 million games for the final result, which means it's going to take me about 1700 years to replicate this. ...
1 vote
0 answers
37 views

Is there a way to beat AlphaGo Zero with different method?

As I read the research from https://deepmind.com/research It seem AlphagoZero use zero knowledge and use Reinforcement learning to improve the ai skill of playing. Is there a way to beat AlphagoZero? ...
3 votes
2 answers
151 views

How does the AlphaGo Zero policy decide what move to execute?

I was going through the AlphaGo Zero paper and I was trying to understand everything, but I just can't figure out this one formula: $$ \pi(a \mid s_0) = \frac{N(s_0, a)^{\frac{1}{\tau}}}{\sum_b N(s_0,...
3 votes
1 answer
132 views

AlphaGo Zero: does $Q(s_t, a)$ dominate $U(s_t, a)$ in difficult game states?

AlphaGo Zero AlphaGo Zero uses a Monte-Carlo Tree Search where the selection phase is governed by $\operatorname*{argmax}\limits_a\left( Q(s_t, a) + U(s_t, a) \right)$, where: the exploitation ...
2 votes
1 answer
180 views

Why is tree search/planning used in reinforcement learning?

In AlphaGo Zero, MCTS is used along with policy networks. Some sources say MCTS (or planning in general) increases the sample efficiency. Assumed the transition model is known and the computational ...
1 vote
1 answer
61 views

Is which sense was AlphaGo "just given a rule book"?

I was told that AlphaGo (or some related program) was not explicitly taught even the rules of Go -- if it was "just given the rulebook", what does this mean? Literally, a book written in ...
2 votes
1 answer
329 views

Did Alphago zero actually beat Alphago 100 games to 0?

tl;dr Did AlphaGo and AlphaGo play 100 repetitions of the same sequence of boards, or were there 100 different games? Background: Alphago was the first superhuman go player, but it had human tuning ...
0 votes
1 answer
151 views

What is the search depth of AlphaGo and AlphaGo Zero?

I cannot find reliable sources but someone says it is 40 moves and someone else says it is 50+ moves. I read their papers and they use value function (NN) and policy function to trim the tree, so more ...
1 vote
1 answer
93 views

How AlphaGo Zero is learning from $\pi_t$ when $z_t = -1$?

I have questions on the way AlphaGo Zero is trained. From original AlphaGo Zero paper, I knew that AlphaGo Zero agent learns a policy, value functions by the gathered data $\{(s_t, \pi_t, z_t)\}$ ...
3 votes
1 answer
282 views

What is the input to AlphaGo's neural network?

I have been reading an article on AlphaGo and one sentence confused me a little bit, because I'm not sure what it exactly means. The article says: AlphaGo Zero only uses the black and white stones ...
2 votes
1 answer
161 views

Why does AlphaGo Zero select move based on exponentiated visit count?

From the AlphaGo Zero paper, AlphaGo Zero uses an exponentiated visit count from the tree search. Why use visit count instead of the mean action value $Q(s, a)$?
4 votes
2 answers
616 views

What part of the game is the value network trained to predict a winner on?

The Alpha Zero (as well as AlphaGo Zero) papers say they trained the value head of the network by "minimizing the error between the predicted winner and the game winner" throughout its many self-play ...
0 votes
0 answers
40 views

How does (or should) AlphaGoZero (which does chess) fare against Deep Blue?

Deep blue is good at chess, but is more "hand-coded" or "top-down". https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer) AlphaGoZero is "self-taught", and at Go is very much super-human. https://...
2 votes
0 answers
148 views

AlphaGo Zero: Does the policy head give a probability for every possible move?

If I understood correctly, the AlphaGo Zero network returns two values: a vector of logit probabilities p and a value v. My question is: in this vector that it is outputted, do we have a probability ...
2 votes
0 answers
74 views

Alphazero Value loss doesn't decrease

Currenly I'm trying to reimplement alphazero in pure c++ using libtorch to accomodate my project's need. But when I training my model, I found out that the value loss doesn't decrese at all after even ...
10 votes
1 answer
995 views

Why is the merged neural network of AlphaGo Zero more efficient than two separate neural networks?

AlphaGo Zero contains several improvements compared to its predecessors. Architectural details of Alpha Go Zero can be seen in this cheat sheet. One of those improvements is using a single neural ...
3 votes
1 answer
153 views

Why is Monte Carlo used as the tree search algorithm for AlphaGo?

Could a better algorithm other than Monte Carlo be used for the AlphaGo computer? Why didn't the DeepMind team think of choosing another kind of algorithm rather than spending time on their neural ...
4 votes
1 answer
258 views

How Does AlphaGo Zero Implement Reinforcement Learning?

AlphaGo Zero (https://deepmind.com/blog/alphago-zero-learning-scratch/) has several key components that contribute to it's success: A Monte Carlo Tree Search Algorithm that allows it to better search ...
2 votes
1 answer
36 views

How do the achievements met in the gaming field (ex. AlphaGo Zero) impact other fields of application?

How can we use the ability of AlphaGo Zero computer, to do something in any other life important related field? Is it possible to make something important besides having created something so smart ...
5 votes
2 answers
2k views

What is the difference between DQN and AlphaGo Zero?

I have already implemented a relatively simple DQN on Pacman. Now I would like to clearly understand the difference between a DQN and the techniques used by AlphaGo zero/AlphaZero and I couldn't find ...
14 votes
1 answer
2k views

Why does the policy network in AlphaZero work?

In AlphaZero, the policy network (or head of the network) maps game states to a distribution of the likelihood of taking each action. This distribution covers all possible actions from that state. ...
7 votes
3 answers
280 views

Would AlphaGo Zero become perfect with enough training time?

Would AlphaGo Zero become theoretically perfect with enough training time? If not, what would be the limiting factor? (By perfect, I mean it always wins the game if possible, even against another ...