Why is a constant plane of ones added into the input features of AlphaGo?

Question

In the paper Mastering the game of Go with deep neural networks and tree search, the input features of the networks of AlphaGo contains a plane of constant ones and a plane of constant zeros, as following.

Feature       #of planes Description 
Stone colour  3          Player stone/opponent stone/empty 
Ones          1          A constant plane filled with 1 
Turns since   8          How many turns since a move was played 
Liberties     8          Number of liberties (empty adjacent points) 
Capture size  8          How many opponent stones would be captured 
Self-atari size 8        How many of own stones would be captured 
Liberties after move 8   Number of liberties after this move is played 
Ladder capture 1         Whether a move at this point is a successful ladder capture 
Ladder escape 1          Whether a move at this point is a successful ladder escape 
Sensibleness  1          Whether a move is legal and does not fill its own eyes 
Zeros         1          A constant plane filled with 0 
Player color  1          Whether current player is black

I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.

What's more, I don't understand the sharp sign here. Does it mean "the number"? But one number is enough to represent "the number of turns since a move was played", why eight?

Thank you very much.

Answer 1

I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.

In many implementations of convolutional layers, the filters do not neatly remain inside the features plane when "sliding" along it, but (conceptually) also partially go "outside" the plane (where always at least one "cell" of the filter will still be inside the plane). Intuitively, in the case of a $3\times3$ filter for example, you can imagine that we pad the input features with an extra border of size $1$, and this padding around the "real" input planes is filled with $0$s.

If it's possible for all input features to also have values of $0$, it may in some situations be difficult or impossible for the neural network to distinguish the "real" $0$ inputs from the $0$ entries in the padding around the board, i.e. it may struggle to know where the game board is and where the game board ends. Having a constant input plane that's always filled with $1$s can help in this respect, because that plane can always reliably be used to distinguish "real" cells that actually exist on the game board from positions that are just outside the game board.

As for the plane filled with $0$s... I have no idea why that would ever be useful. Maybe it was useful due to some peculiar implementation detail. In this thread, some people hypothesise that on specific hardware it might make some computations slightly more efficient because of the layout of the data in memory -- it causes the number of channels to be divisible by $8$, which will.. maybe help? I really don't know too much about this, but I do know that on a smaller scale, sometimes adding unused data in classes can indeed increase performance due to better layout in memory. I suppose it's also possible that it was accidentally added by mistake, or "just in case" and that it doesn't really have much of a purpose. The amount of hardware that the AlphaGo team had available to them is quite insane anyway, one channel more or less probably wasn't too big of a concern for them.

---

What's more, I don't understand the sharp sign here. Does it mean "the number"? But one number is enough to represent "the number of turns since a move was played", why eight?

This is explained in the "Features for policy/value network" paragraph in the paper. Quote from the paper:

"Each integer feature value is split into multiple $19 \times 19$" planes of binary values (one-hot encoding). For example, separate binary feature planes ares used to represent whether an intersection has $1$ liberty, $2$ liberties, $\dots$, $\geq 8$ liberties.

All feature planes used were strictly binary, no feature planes were used that had integer values $> 1$. This is quite common when possible, because neural networks tend to have a much easier time learning with binary features than with integer- or real-valued features.