1
$\begingroup$

AlphaZero utilized a residual convolutional neural network to estimate move policy and position value. If it was rebuilt today, would it be more efficient and powerful if they used a transformer architecture instead? Assume the same amount of compute power and training time is available, and purely self-play, aiming for as fair a comparison as possible.

$\endgroup$

1 Answer 1

4
$\begingroup$

Yes!

LeelaChessZero, an open-source re-implementation and continuation of AlphaZero, has been experimenting with this for a while now. Their strongest networks are currently transformers, not convolution-based ResNets anymore.

Some resources:

$\endgroup$
2
  • $\begingroup$ Cool! Does Leela still use MCTS, in conjunction with the new transformer architecture? $\endgroup$
    – Ben G
    Feb 24 at 18:20
  • $\begingroup$ Yeah they do. They've added some improvements to the core UCT formula and to the selfplay data generation process, but the big picture is still very much the same. $\endgroup$ Feb 24 at 18:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .