2
$\begingroup$

I've written a Monte Carlo Tree Search player for the game of Castle (AKA Shithead, Shed, Palace...). I have set this MCTS player to play against a basic rule-based AI for ~30000 games and collected ~1.5 million game states (as of now) along with whether the MCTS player won that particular game in the end after being in that particular game state. The game has a large chance aspect, and, currently, the MCTS player is winning ~55% of games. I want to see how high I can push it. In order to do this, I aim to produce a NN that will act as a game state evaluation function to use within the MCTS.

With this information, I've already tried an SVM, but came to the conclusion that the game space is too large for the SVM to classify a given state accurately.

I hope to be able to train a NN to evaluate a given state and return how good that state is for the MCTS player. Either with a binary GOOD/BAD or I think it would be more helpful to return a value between 0-1.

The input to the NN is a $4 \times 41$ NumPy array of binary values (0, 1) representing the MCTS players hand, MCTS face-up cards, OP face-up cards, MCTS no. face-down cards, OP no. face-down cards. Shown below.

enter image description here

Describes the np.array:

enter image description here

The np.array is made from the database entries of game states. An example of this information is below. However, I am currently omitting the TOP & DECK_EMPTY columns in this model. WON (0, 1) is used as the label.

enter image description here

This is my keras code:

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(2, activation=tf.nn.softmax))

model.compile(optimizer='adam',
             loss='sparse_categorical_crossentropy',
             metrics=['accuracy'])

model.fit(X_train, y_train, epochs=3)

This model isn't performing.

  • Do you think it is possible to obtain a useful NN with my current approach?

  • What layers should I look to add to the NN?

  • Can you recommend any training/learning material that I could use to try and get a better understanding?

Improve this question
$\endgroup$
0

0

Reset to default

You must log in to answer this question.