Did Alphago zero actually beat Alphago 100 games to 0?

Question

tl;dr
Did AlphaGo and AlphaGo play 100 repetitions of the same sequence of boards, or were there 100 different games?

Background:
Alphago was the first superhuman go player, but it had human tuning and training.

AlphaGo zero learned to be more superhuman than superhuman. Its supremacy was shown by how it beat AlphaGo perfectly in 100 games.

My understanding of AlphaGo and AlphaGo are that they are deterministic, not stochastic.

If they are deterministic, then given a board position they will always make the same move.

The way that mathematicians count the possible games in chess is to account for different board positions. As I understand it, and I could be wrong, if they have the exact same sequence of board positions then it does not count as a different game.

If they make the same sequence of moves 100 times, then they did not play 100 different games, but played one game for 100 repetitions.

Question:
So, using the mathematical definition, did AlphaGo and AlphaGo Zero play only one game for 100 iterations or did they play 100 different games?

References:

• https://www.scientificamerican.com/article/ai-versus-ai-self-taught-alphago-zero-vanquishes-its-predecessor/
• https://deepmind.com/blog/article/alphago-zero-starting-scratch
• https://mathworld.wolfram.com/Stochastic.html

Answer 1

Did AlphaGo and AlphaGo [Zero] play 100 repetitions of the same sequence of boards, or were there 100 different games?

There were 100 different games. You can view some example games between AlphaGo [Lee] and AlphaGo Zero here. They are clearly all different.

This statement in the question shows a misunderstanding:

My understanding of AlphaGo and AlphaGo [Zero] are that they are deterministic, not stochastic.

The Monte Carlo Tree Search (MCTS) algorithm used for look-ahead planning in AlphaGo and Alpha Zero is inherently stochastic. It samples from the huge tree of possibilities in a game like Go by making weighted random choices at all branch points. That means play can progress stochastically with two such agents opposing each other, as many board states will resolve into selecting semi-randomly between "best" moves that would be very closely ranked by each agent in the limit of very long search times.

Whilst this solves the main point of your question, it is worth noting that there can be a related effect in self-play algorithms, even if they are partially stochastic. That is, it is possible to have one agent that develops a specific weakness by chance, that another agent consistently takes advantage of, such that agent A consistently beats agent B, and wins in a very similar fashion each time (maybe deterministically, maybe across a range of different games all with a similar mistake). However it may be the case that also:

• Neither agent is strong in general.

• Another agent C can beat B consistently, but will lose to A consistently. There would then be no clear way to rank agents A, B, and C without further measurements.

Agents trained through self play therefore do need to be trained and tested against a wide range of opponents to verify this is not happening and that the skill level assessment is valid more generally. I believe this was done with all the AlphaGo variants built by DeepMind.

The MCTS algorithm does help a little with this scenario as it can correct for weaknesses in how a trained neural network rates early board positions. The look-ahead planning of MCTS makes initial ratings less relevant to eventual action selection - effectively it refines those learned ratings using the samples from current position.