Questions tagged [alphago-zero]

For questions related to AlphaGo Zero, which is a version of DeepMind's Go software, AlphaGo, that does not use data from human games and it is stronger than AlphaGo. There is a generalized version of AlphaGo Zero called AlphaZero, which beat the 3-day version of AlphaGo Zero by winning 60 games to 40. AlphaGo Zero was introduced in the paper "Mastering the game of Go without human knowledge" (2017) by David Silver et al.

Filter by
Sorted by
Tagged with
0
votes
1answer
41 views

What is the search depth of AlphaGo and AlphaGo Zero?

I cannot find reliable sources but someone says it is 40 moves and someone else says it is 50+ moves. I read their papers and they use value function (NN) and policy function to trim the tree, so more ...
1
vote
1answer
51 views

In Alpha(Go)Zero, why is the policy extracted from MCTS better than the network one?

I've read through the Alpha(Go)Zero paper and there is only one thing I don't understand. The paper on page 1 states: The MCTS search outputs probabilities π of playing each move. These search ...
1
vote
1answer
78 views

How AlphaGo Zero is learning from $\pi_t$ when $z_t = -1$?

I have questions on the way AlphaGo Zero is trained. From original AlphaGo Zero paper, I knew that AlphaGo Zero agent learns a policy, value functions by the gathered data $\{(s_t, \pi_t, z_t)\}$ ...
0
votes
0answers
26 views

Structure for neural network

My purpose is to apply deep learning for planning. To do so, I decided to use a similar approach as AlphaGo. But my "game state" is very different. Instead of considering some planes ...
0
votes
0answers
55 views

Total loss increasing, but loss components are decreasing?

I am writing a AlphaGo Zero clone, and sometimes in the training the policy head loss and value head loss would both be decreasing, but the total loss is increasing? How is this possible? I am using ...
3
votes
1answer
74 views

What is the input to AlphaGo's neural network?

I have been reading an article on AlphaGo and one sentence confused me a little bit, because I'm not sure what it exactly means. The article says: AlphaGo Zero only uses the black and white stones ...
2
votes
1answer
65 views

Why does AlphaGo Zero select move based on exponentiated visit count?

From the AlphaGo Zero paper, AlphaGo Zero uses an exponentiated visit count from the tree search. Why use visit count instead of the mean action value $Q(s, a)$?
1
vote
1answer
63 views

How does the AlphaGo Zero policy decide what move to execute?

I was going through the AlphaGo Zero paper and I was trying to understand everything, but I just can't figure out this one formula: $$ \pi(a \mid s_0) = \frac{N(s_0, a)^{\frac{1}{\tau}}}{\sum_b N(s_0,...
1
vote
1answer
199 views

How should I interpret the weights file of the Leela Zero neural network?

I am trying to understand the NN (Neural Network) architecture given at https://github.com/leela-zero/leela-zero/blob/next/training/caffe/zero.prototxt. So, I downloaded the NN weights from https://...
4
votes
2answers
482 views

What part of the game is the value network trained to predict a winner on?

The Alpha Zero (as well as AlphaGo Zero) papers say they trained the value head of the network by "minimizing the error between the predicted winner and the game winner" throughout its many self-play ...
0
votes
0answers
31 views

How does (or should) AlphaGoZero (which does chess) fare against Deep Blue?

Deep blue is good at chess, but is more "hand-coded" or "top-down". https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer) AlphaGoZero is "self-taught", and at Go is very much super-human. https://...
2
votes
0answers
52 views

AlphaGo Zero: Does the policy head give a probability for every possible move?

If I understood correctly, the AlphaGo Zero network returns two values: a vector of logit probabilities p and a value v. My question is: in this vector that it is outputted, do we have a probability ...
3
votes
0answers
115 views

How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?

In a vanilla Monte Carlo tree search (MCTS) implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only ...
2
votes
0answers
46 views

Alphazero Value loss doesn't decrease

Currenly I'm trying to reimplement alphazero in pure c++ using libtorch to accomodate my project's need. But when I training my model, I found out that the value loss doesn't decrese at all after even ...
10
votes
1answer
867 views

Why is the merged neural network of AlphaGo Zero more efficient than two separate neural networks?

AlphaGo Zero contains several improvements compared to its predecessors. Architectural details of Alpha Go Zero can be seen in this cheat sheet. One of those improvements is using a single neural ...
3
votes
1answer
99 views

Why is Monte Carlo used as the tree search algorithm for AlphaGo?

Could a better algorithm other than Monte Carlo be used for the AlphaGo computer? Why didn't the DeepMind team think of choosing another kind of algorithm rather than spending time on their neural ...
3
votes
1answer
145 views

How Does AlphaGo Zero Implement Reinforcement Learning?

AlphaGo Zero (https://deepmind.com/blog/alphago-zero-learning-scratch/) has several key components that contribute to it's success: A Monte Carlo Tree Search Algorithm that allows it to better search ...
2
votes
1answer
31 views

How do the achievements met in the gaming field (ex. AlphaGo Zero) impact other fields of application?

How can we use the ability of AlphaGo Zero computer, to do something in any other life important related field? Is it possible to make something important besides having created something so smart ...
5
votes
2answers
905 views

What is the difference between DQN and AlphaGo Zero?

I have already implemented a relatively simple DQN on Pacman. Now I would like to clearly understand the difference between a DQN and the techniques used by AlphaGo zero/AlphaZero and I couldn't find ...
10
votes
1answer
1k views

Why does the policy network in AlphaZero work?

In AlphaZero, the policy network (or head of the network) maps game states to a distribution of the likelihood of taking each action. This distribution covers all possible actions from that state. ...
6
votes
3answers
209 views

Would AlphaGo Zero become perfect with enough training time?

Would AlphaGo Zero become theoretically perfect with enough training time? If not, what would be the limiting factor? (By perfect, I mean it always wins the game if possible, even against another ...
1
vote
0answers
60 views

How basically AI synthesize thinking?

In the instance like AlphaGo Zero. How N' why AlphaGo Zero's training is so stable? Compared with traditional game theory applied in Deep RN technique!? How "AlphaGo Zero" differs from "...