2
$\begingroup$

There is a quite popular video analysing a chess game AlphaZero vs. AlphaZero, called "the perfect game". It leaves some questions open and I'd like to ask them here:

  1. Did the two copies of AlphaZero use the same random seeds in the learning phase, esp. when generating random games? So were they perfect copies of each other at the beginning of the game?

  2. Do the two copies of AlphaZero calculate their moves with the same random seeds when doing Monte Carlo tree search?

In any case there are three possible games all of which may be called "AlphaZero vs. AlphaZero":

  1. same seeds when learning, same seeds when playing
  2. same seeds when learning, different seeds when playing
  3. different seeds when learning (the seeds when playing don't matter then)

In the latter case, I wonder if AlphaZero's playing style (which is for example analyzed in the book Game Changer by Matthew Sadler and Natasha Regan) may depend on the seeds used for generating random games (assuming that the same number of test games is played in the learning phase). In other words: Can AlphaZero develop significantly different playing styles when starting tabula rasa? In this case, Sadler/Regan's book would describe just one instance of AlphaZero.

The last question should be answerable also by those of you who don't know the video and how the premises have been.

$\endgroup$
7
  • $\begingroup$ Can you please put your specific question in the title? "AlphaZero vs. AlphaZero" is not a question and is it's not specific. $\endgroup$
    – nbro
    Feb 24, 2022 at 13:53
  • 1
    $\begingroup$ @nbro: Done. Sorry again, I will learn it;-) $\endgroup$ Feb 24, 2022 at 15:57
  • 1
    $\begingroup$ I'm fairly certain that it's just the same copy of alpha zero, playing against itself. $\endgroup$
    – Taw
    Feb 26, 2022 at 7:39
  • 1
    $\begingroup$ The strength of alpha zero is proportional to how much it has been trained; so if you had two copies of it that were trained for a different number of episodes, or two copies which used different hyperparameters, they may end up at different ELO strengths. But it's awfully hard to imagine that they would have different playing styles, except perhaps at very low, sub human elo due to randomness from initialization $\endgroup$
    – Taw
    Feb 26, 2022 at 7:41
  • $\begingroup$ @Taw: Thanks a lot, this helps. Can you tell me where I can find a concise list of AlphaZero's hyperparameters? In the original paper only parameters $\theta$ are mentioned (leaving open which exactly these are), and there is no talk of hyperparameters. Could you please clarify? $\endgroup$ Feb 26, 2022 at 10:16

1 Answer 1

1
$\begingroup$

The primary questions

  1. They only trained AlphaZero once, and then let it play against itself. So yes, both players are identical.

  2. MCTS as implemented by AlphaZero does not intentionally use any randomness during tournament play. There is probably a bit of inherent randomness from multithreading, but no seed as such. I talk about this is a bit more in this answer and its comments.

Different A0 vs A0 games

Something you seem to be missing is that typically when engine vs engine matches are played an opening book is used. This is mostly to ensure game diversity, since many engines are (almost) deterministic, so you'd just end up with a single game. The video you linked links to a chess.com article, with some more details:

  • Computer Match, selfplay from move 4 pg

  • 1. e4 c62 2. d4 d5 the last book move

So the first 4 moves of the game where set by humans, only afterwards AlphaZero started to play by itself. This means you can get as many different games as you want, in fact at the bottom of the article there are a couple more with different opening moves.

Different playing styles for different seeds

This is hard to answer conclusively, since "playing styles" are a bit of a nebulous concept. I've trained smaller scale versions of A0 myself, and after training for a while they always end up around the same playing strength for the same hyperparameters.

Intuitively I think that different network initialization seeds would not really cause different playing styles, or at least that any differences disappear after a training for a while.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .