4
$\begingroup$

I am currently writing an engine to play a card game and I would like for an ANN to learn how to play the game. The game is currently playable, and I believe for this game a deep-recurrent-Q-network with a reinforcement learning approach is the way to go.

However, I don't know what type of layers I should use, I found some examples of Atari games solved through ANN, but their layers are CNN (convolutional), which are better for image processing. I don't have an image to feed the NN, only a state composed of a tensor with cards in the player's own hand and cards on the table. And the output of the NN should be a card or the action 'End Turn'.

I'm currently trying to use TensorFlow but I'm open to any library that can work with NN. Any type of help or suggestion would be greatly appreciated!

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ I encourage you to use some high level framework like Keras to get the work done easily. It uses Tensorflow (or Theano) as backend but it is almost transparent. The type of layer depend on you representation of the features. You should concern not only about the type of layers but also about the entire network architecture. Anyway as a baseline, I'll try with a sequential model of full conected layers combining layers with relu and sigmoid activations. $\endgroup$
    – Marcelo Fornet
    Jan 19, 2018 at 16:25
  • $\begingroup$ Hey, thanks a lot for the feedback! I'll check Keras and I'll try to implement that idea and I'll try to reply soon. $\endgroup$
    – Paulo Neves
    Jan 20, 2018 at 16:31

2 Answers 2

Reset to default
2
$\begingroup$

The game state consists of the location of all the hidden cards, so you probably need a softmax layer, 52*n, where n is the number of locations.

I'm not very sure that a NN is a good match.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the reply! I had a typo saying the "state composed of a tensor with cards in the players hand", I meant in the player's own hand. The last layer could indeed be a softmax activation layer. Once i'm able to test it, I'll reply (i'm still having problems with the rest of the architecture). I tried 2 fully connected layers with relu and sigmoid activation and curiously got a worse result than with a simple weights matrix. $\endgroup$
    – Paulo Neves
    Jan 23, 2018 at 14:33
1
$\begingroup$

With images, you can use CNN because of the translational invariance. A filter which is good in one area will probably be good in another area, too.

With images, you must use CNN because otherwise, there would be too many weights to train.

With your game, it depends on the representation and the exact rules. Note that Alpha Zero uses a set of 19 x 19 inputs with CNN for playing Go.

In a game like Bridge, where each card has its color and rank, there's a kind of translational invariance. Having Ace and Queen is a bit similar to having King and Jack - in both cases you have a 50% chance of catching the card in between. At the same time, the strengths of AQ and KJ are very different, so a pure CNN would improbably work well.

The more important symmetry is the one among colors. After the auction, there's one or none trump color and all other colors are equivalent. This probably means that the corresponding weights should be the same.

In some card games, many cards are special and there's no symmetry at all. You didn't tell us anything about your game, so it's hard to give a more concrete advice.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ (Part 1 Comment) Thanks for your response. Before I learned more about NN I though they would be more generalist. The game is not too complicated, there can be an N number of players, to which all cards in the deck are distributed in the beginning. For example 4 players mean each will start with 13 cards, and the winner is the first to have no cards in hand. To get rid of the cards they will play them in turn order and can choose to play cards or not play. The only rules are that the players always have to play higher cards than previously played. $\endgroup$
    – Paulo Neves
    Mar 27, 2018 at 12:47
  • 1
    $\begingroup$ (Part 2 of the Comment) Once no one wants to play a card (or can't), the last player to play a card gets to play any card he wants and start a new cycle. The game is highly stochastic because the players can start with low level cards and lose the game even if they played better than the opposition, which complicates the use of greedy epsilon exploration. $\endgroup$
    – Paulo Neves
    Mar 27, 2018 at 12:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .