4
$\begingroup$

Recently, DeepMind's AlphaZero chess algorithm did better than the prior best chess software Stockfish. I read the paper Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm about it, but I'm not sure if:

  • is there a value given for each piece (e.g. 1 for pawn, 3 for the knight, 9 for the queen, etc.) to train the algorithm, or does the algorithm learn this by himself?

  • I read that the algorithm uses Monte Carlo Tree Search, but what are the key improvements to prior chess algorithms already using MCTS?

  • Is there hope for being able to run it an average computer? They said it required 9 hours learning (starting with nearly 0 knowledge except for rules (and maybe value for a piece?)), and 24 millions of games. Is it something doable in maybe 1 month with an average computer?

$\endgroup$
  • 2
    $\begingroup$ Too many questions in one post. Ideally, one post should contain one question (or a few but very related questions), but you are asking distinct questions, even though in the same context. I closed this post as "too broad", so that to avoid getting other general and not detailed answers (to each of your questions). Please, next time, ask one question per post. $\endgroup$ – nbro Dec 26 '17 at 14:08
2
$\begingroup$

is there a value given for each piece (e.g. 1 for pawn, 3 for knight, 9 for queen, etc.) to train the algorithm, or does the algorithm learn this by himself?

No, there are no such explicit values assigned to pieces, no manually-constructed evaluation functions. The paper states that "no domain knowledge" is given to the algorithm other than the game's rules (necessary to run simulations / run search algorithms like MCTS).

I read that the algorithm uses Monte Carlo Tree Search, but what are the key improvements to prior chess algorithms already using MCTS?

The key improvements are in the way that Deep Learning (deep neural networks), Reinforcement Learning, and self-play is combined with MCTS. This is quite similar to the methods previously used by AlphaGo Zero in the game of Go as well. There (likely) have been combinations of Deep Learning + MCTS before (e.g. using a learned Neural Network to bias rollouts in MCTS), but the specific way in which they are combined in AlphaZero is critical (specifically, using the "search behaviour" of MCTS as one of the training signals for the Neural Network). It probably also helped that we're talking about Google here, who could afford to use thousands of Tensor Processing Units (TPUs) for training.

Is there a hope for being able to run it an average computer? They said it required 9 hours learning (starting with nearly 0 knowledge except rules (and maybe value for piece?)), and 24 millions of games. Is it something doable in maybe 1 month with a average computer?

Based on the information in the paper, I highly doubt this. As mentioned above, training was done with thousands of TPUs (specifically; 5,000 first-generation TPUs and 64 second-generation TPUs) in parallel. One month is only about a factor 80 times greater than the 9 hours training time reported in the paper, whereas all of those TPUs is much more than 80 average computers.

After training, it likely could run on a fairly average computer (or maybe high-end computer) to play though. But it'll need much more power to train first.

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.