Reason why chess neural network might not be training

Question

I've been trying to use a Stockfish-like chess evaluation neural network for the past few weeks but to no avail. I wanted to get some other opinions about why my current methods haven't worked.

Input: $8 \times 8 \times 12$ one-hot encoded board of pieces. I have around ~60 million unique examples. Evaluations are placed between -15 and 15 (anything greater or smaller is brought to 15 or -15)

Output: Single number representing Stockfish evaluation.

I've tried both fully connected and convolutional models and neither have worked very well. Here's some Tensorflow code that might give an idea of what the structures looked like:

Convolutional model:

import tensorflow as tf

def block(filters, x, res=None):
  if res is not None:
    x = tf.keras.layers.concatenate([x, res])
  x = tf.keras.layers.Conv2D(filters=filters, kernel_size=3, padding='same', activation='relu', kernel_initializer='he_normal')(x)
  x = tf.keras.layers.BatchNormalization()(x)
  x = tf.keras.layers.MaxPooling2D(pool_size=2, strides=1, padding='same')(x)
  
  return x

board = tf.keras.Input(shape=(8, 8, 12))
conv1 = block(64, board)
conv2 = block(128, conv1)
conv3 = block(256, conv2)
conv4 = block(512, conv3, conv3)
conv5 = block(1024, conv4, conv2)
conv6 = block(1024, conv5, conv1)
conv7 = block(512, conv6)
x = tf.keras.layers.Flatten()(conv7)
x = tf.keras.layers.Dense(128, activation='relu')(x)
x = tf.keras.layers.Dense(32, activation='relu')(x)
x = tf.keras.layers.Dense(1)(x)

Fully connected model:

import tensorflow as tf

board = tf.keras.Input(shape=(8, 8, 12))
x = tf.keras.layers.Flatten()(board)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(256, activation='relu')(x)
x = tf.keras.layers.Dense(1)(x)

Just to add a few notes about what the current results are, they are decent (e.g. mean squared error of Stockfish evaluated cut off between -15 and 15 is about 7). The early game is also played fairly decently but the late game gets really bad (the engine can't evaluate that bishop takes enemy queen is good). My evaluation network also evaluates the four move checkmate as black favoured.

Answer 1

It could be because there is simply not enough data for the late game. To make the model give more importance to the later stages of the game you can try to tweak the loss function such that it penalizes more for when there are fewer pieces on the board. ( This might give an idea on that: https://medium.com/visionwizard/understanding-focal-loss-a-quick-read-b914422913e7)

One other thing you can do; given that my above-mentioned reasoning is correct, is to provide the number of pieces on the board as an input. I am not sure how you might go around doing that in the one-hot encoded scenario but might be worth looking into (You could append the number of pieces in the binary format to the flattened input). One argument against this is that the network should implicitly know how many pieces are on the board but yet it could still be useful to give it as an extra input.

Answer 2

I tried hacking together several NN approaches for chess, in the context of Chess Coding Challenge (implement a chess bot in a relatively small amount of code). I also play online chess semi-regularly, so I have some understanding of the game. I found that in general the more accurate my network was with evaluation (and the more complex it was), the worse it played since it ended up in uncommon situations which it grossly mis-evaluated to its favor.

I haven't done a detailed analysis, but I suspect that there two main factors here:

• Lack of diversity in the training data, for example does your network correctly evaluate a starting position with queen odds?
• The difficulty of evaluating positions which involve tactics

Here are three similar positions, sorry it is always black's turn to move, since I encountered these when playing as black:

Evaluation: -6.4

Evaluation: -3.3

Evaluation: -2.2

I don't know a good solution on how to correctly evaluate tactics, except searching explicit steps in the game tree. Note the first position has a rather simple two-move tactic, but modern engines will do searches tens of moves deep.

And regarding the first issue, I got better results by imposing strong regularization to the network's architecture and weights. For example a good starting point would be an evaluation function which resembles a Piece-Square-Table.