# Tag Info

11

Good question! AlphaZero, though a major milestone, is most definitely not an AGI :) AlphaGo, though strong at the game of Go, is narrowly strong ("strong-narrow AI"), defined as strength in a single problem or type of problem (such as Go and other non-chance, perfect information games.) AGI, at minimum, must be about as strong as humans in all problems ...

7

There are at least two questions in your question: What are some of the methods used to program the successful go playing program? and Are those methods considered to be artificial intelligence? The first question is deep and technical, the second broad and philosophical. The methods have been described in: Mastering the Game of Go with Deep Neural ...

6

It doesn't make much sense to have a single threshold with "unintelligent" below it and "intelligent" above it. I think it makes more sense to have a gradation of intelligence by cognitive task. Inverting a matrix is a 'cognitive task,' and one where working memory pays off immensely; computers have been much better at that cognitive task than humans for a ...

5

Did AlphaGo and AlphaGo [Zero] play 100 repetitions of the same sequence of boards, or were there 100 different games? There were 100 different games. You can view some example games between AlphaGo [Lee] and AlphaGo Zero here. They are clearly all different. This statement in the question shows a misunderstanding: My understanding of AlphaGo and AlphaGo [...

5

We know what Lee's strategy was during the game, and it seems like the sort of thing that should work. Here's an article explaining it. Short version: yes, we know what went wrong, but probably not how to fix it yet. Basically, AlphaGo is good at making lots of small decisions well, and managing risk and uncertainty better than humans can. One of the things ...

4

Assuming you mean a mathematically perfect player, similar to what we can achieve trivially in Tic Tac Toe, then the answer is "maybe". The underlying reinforcement learning algorithms that it uses do have some convergence guarantees, but there are some caveats: Theories of convergence that apply to value and policy functions learned by RL assume ...

3

The technique used by AlphaGo is "Monte Carlo Tree Search", combined with a very well trained neural network. The network's job is to estimate the quality of different board states and moves. This estimation is deterministic. If you show AlphaGo the same board on two different occasions, it thinks it is exactly as good (or bad) on both occasions. Monte ...

3

To my understanding, this is basically a supervised learning problem, where from the self play we have games associated with their winners, and the network is being trained to map game states to likelihood of winning. Yes, although the data for this supervised learning problem was provided by self-play. As AlphaZero learned, the board evaluations of the ...

3

Now that this milestone has been reached, does that represent a significant advance in artificial intelligence techniques or was it just a matter of ever more processing power being applied to the problem? Neither, really. It is a milestone and a significant advance in computers beating humans in games, but the techniques used are only relevant to that game,...

3

We cannot tell with certainty whether AlphaGo Zero would become perfect with enough training time. This is because none of the parts (Neural Network) that would benefit from infinite training time (= a nice approximation of "enough" training time) are guaranteed to ever converge to a perfect solution. The main limiting factor is that we do not know whether ...

2

We've had many discussions on what constitutes Artificial Intelligence, and my takeaway has been that decision-making is the core requirement of AI, regardless of the optimality of that decision. In this conception, Nimatron (1939, US2215544A) might be thought of as the first proper AI, pending verification of a a fabled Babbage Tic-Tac-Toe machine. ...

2

Is it fair to compare AlphaGo with a Human player? Depends on the purpose of the comparison. If we are comparing ability to win a game of Go, then yes. If we are comparing learning ability, then maybe. It depends on the task. AlphaGo and systems like it are capable of learning only in well-described limited domains. There may be an analogy with sensory ...

2

Assumptions That May Be Incorrect There are two assumptions identifiable in the tone of the paper. All mental challenges can be reduced to a game with fixed rules. Machines better than humans are what humans really want or need. There is another two identifiable in the question. General intelligence exists in humans1 If it exists in humans, it is ...

2

The loss of the policy head here is really quite different from losses in, for instance, more "conventional" Supervised Learning approaches (where we typically expect/hope to see a relatively steady decrease in loss function). In this AlphaZero setup, the target that we're updating the policy head towards is itself changing during the training process. When ...

1

For easier visualization, I recommend this video: https://twitter.com/i/status/1257053365424578565 The more detailed article about GO algorithms: https://deepmind.com/blog/article/alphago-zero-starting-scratch. With its breadth of $250$ possible moves each turn (go is played on a $19$ by $19$ board, compared to the much smaller $8$ by $8$ chess field) and a ...

1

The formula in question uses a function N(state, action) that defines a visit count of a state-action pair (introduced on page 3). To describe how it is used, lets first describe the steps of AlphaGo Zero as a whole. There are 4 "phases" to the Monte-Carlo tree search in AlphaGo Zero as depicted in Figure 2. The first 3 expand and update the tree and ...

1

The paper that introduced AlphaGo, Mastering the game of Go with deep neural networks and tree search, motivates the use of MCTS Monte Carlo tree search (MCTS) uses Monte Carlo rollouts to estimate the value of each state in a search tree. As more simulations are executed, the search tree grows larger and the relevant values become more accurate. The ...

1

The algorithm in the 2012 survey article (your second link) is the most common / standard implementation. Whenever someone mentions using MCTS or UCT, without explicitly stating any other info, it's safe to assume that that pseudocode is what they're using. The paper by Kocsis and Szepesvári from 2006 (your first link) is (one of) the original publication(s)...

1

I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train. In many implementations of convolutional layers, the filters do not neatly remain inside the features plane when "sliding" along it, but (conceptually) also partially go "outside" the plane ...

1

Yes AlphaGo Zero could become undeniably perfect. It has won 100:0 against AlphaGo Lee (which won 4:1 against 18-time world champion (human) Lee Sedol) and 89:11 against AlphaGo Master (which won 60 straight online games against human professional Go players from 29 December 2016 to 4 January 2017). From the official AlphaGo website: "AlphaGo's 4-1 ...

Only top voted, non community-wiki answers of a minimum length are eligible