I've been trying to use a Stockfish-like chess evaluation neural network for the past few weeks but to no avail. I wanted to get some other opinions about why my current methods haven't worked.
Input: $8 \times 8 \times 12$ one-hot encoded board of pieces. I have around ~60 million unique examples. Evaluations are placed between -15 and 15 (anything greater or smaller is brought to 15 or -15)
Output: Single number representing Stockfish evaluation.
I've tried both fully connected and convolutional models and neither have worked very well. Here's some Tensorflow code that might give an idea of what the structures looked like:
Convolutional model:
import tensorflow as tf
def block(filters, x, res=None):
if res is not None:
x = tf.keras.layers.concatenate([x, res])
x = tf.keras.layers.Conv2D(filters=filters, kernel_size=3, padding='same', activation='relu', kernel_initializer='he_normal')(x)
x = tf.keras.layers.BatchNormalization()(x)
x = tf.keras.layers.MaxPooling2D(pool_size=2, strides=1, padding='same')(x)
return x
board = tf.keras.Input(shape=(8, 8, 12))
conv1 = block(64, board)
conv2 = block(128, conv1)
conv3 = block(256, conv2)
conv4 = block(512, conv3, conv3)
conv5 = block(1024, conv4, conv2)
conv6 = block(1024, conv5, conv1)
conv7 = block(512, conv6)
x = tf.keras.layers.Flatten()(conv7)
x = tf.keras.layers.Dense(128, activation='relu')(x)
x = tf.keras.layers.Dense(32, activation='relu')(x)
x = tf.keras.layers.Dense(1)(x)
Fully connected model:
import tensorflow as tf
board = tf.keras.Input(shape=(8, 8, 12))
x = tf.keras.layers.Flatten()(board)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(256, activation='relu')(x)
x = tf.keras.layers.Dense(1)(x)
Just to add a few notes about what the current results are, they are decent (e.g. mean squared error of Stockfish evaluated cut off between -15 and 15 is about 7). The early game is also played fairly decently but the late game gets really bad (the engine can't evaluate that bishop takes enemy queen is good). My evaluation network also evaluates the four move checkmate as black favoured.