I've written an Monte Carlo Tree Search player for the game of Castle (AKA Shithead, Shed, Palace...). I have set this MCTS player to play against a basic rule based AI for ~30000 games and collected ~1.5 million gamestates (as of now) along with whether the MCTS player won that particular game in the end after being in that particular game state. The game has a large chance aspect and currently the MCTS player is winning ~55% of games. I want to see how high I can push it. In order to do this I aim to produce a CNN that will act as a gamestate evaluation function to use within the MCTS.

With this information, I've already tried an SVM but came to the conclusion that the game space is too large for the SVM to classify a given state accurately.

I hope to be able to train a NN to evaluate a given state and return how good that state is for the MCTS player. Either with a binary GOOD/BAD or I think it would be more helpful to return a value between 0-1.

The input to the NN is a 4x41 np.array of binary values (0, 1) representing the MCTS players hand, MCTS face-up cards, OP face-up cards, MCTS no. face-down cards, OP no. face-down cards. Shown below.

enter image description here

Describes the np.array: enter image description here

The np.array is made from the database entries of gamestates. An example of this information is below. However, I am currently omitting the TOP & DECK_EMPTY columns in this model. WON (0, 1) is used as the label. enter image description here

This is my keras code:

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(2, activation=tf.nn.softmax))


model.fit(X_train, y_train, epochs=3)

This model isn't performing.

  • Do you think it is possible to obtain a useful NN with my current approach?
  • What layers should I look to add to the NN?
  • Can you recommend any training/learning material that I could use to try and get a better understanding?

Hope this isn't too broad...

I'm a final year CS student nearing the end (ish) of my final year project and feeling pretty out of my depth at this point..

Any help is greatly appreciated.

  • $\begingroup$ @NeilSlater I'm converting text data describing the game states into 4x41 binary np.array's and using that as the NN input. I have edited the question to try and make this clearer. $\endgroup$ – Cyrus Dobbs Apr 3 at 12:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.