3
$\begingroup$

In order to learn about DP and RL, I chose to start a side project where I would train an AI to play a "simple" card game. I will be doing this using the DQN with replay memory.
The problem is, I can't get the intuition behind how to represent the input to the neural network..

About the game

It's a fairly simple 2-players game. There is a deck of 40 unique cards (4 types of cards, 10 numbered cards in each type).
Each player gets 4 cards and each turn a player must put a card on the table.
If a player puts a card and there is already a card with the same number on the table, the player wins both cards.
If for example a player plays Card 2 and on the table there is Cards 2, 3, 4, 5 then the player wins all those cards (sequence).
Cards won don't go back to the hand nor to the deck, they are just kept as like a score.
When the players have 0 cards in hand, another 4 cards are dealt to each one untile the deck has 0 cards left where then we decide who won based on the number of cards eaten/won.

Question

As the input, I will be using the following:

  • Current cards in the AI hand (40 one-hot-encoded features?)
  • Current cards on the table (40 one-hot-encoded features?)
  • History of played cards (40 one-hot-encoded features?)

This would give 120 columns/features in each state.
I am wondering wheter this is too much for the NN or wheter my input representation would be bad for the NN?
Should the features be represented as a (120,) vector or as a 3x40 matrix?

I am also wondering if it's a good idea to represent the current cards on the table as just a 10 one-hot-encoded features since the type of the cards don't matter and the same number can't exist 2 times in the table?

Thank you in advance.

$\endgroup$
3
$\begingroup$

120 inputs can be handled by a complex enough network. Dealing with high complexity is one of NN's strengths.

Using a (120,) vector or a (3,40) matrix is the same, they're still 120 inputs. Your binary encoding should work. Another option is a single (40,) vector, with 0 being "still in deck", 1 being "in hand", 2 being "on table", 3 being "already played".

If the types of cards are irrelevant, you could actually have a (3,10) matrix with counters of cards (1 in hand, 1 in table, 2 already played). You can try different approaches and see what works best.

$\endgroup$
  • $\begingroup$ Never knew that we can encode features as non binary... Can you tell me the name of this "technique"? Thank you for the informations. $\endgroup$ – Haytam Jul 6 '18 at 18:12
  • $\begingroup$ For the (40,) vector, wouldn't encoding like that mean that there is some kind of order? $\endgroup$ – Haytam Jul 6 '18 at 18:30
  • $\begingroup$ Imagine you had one ace in hand, one on table, 2 played already. How could you code that with a single (10,) vector? You need 3 rows, one for each place. Its simply called non binary input, its very common. Image processing NNs use inputs with pixel values [0,255] $\endgroup$ – BlueMoon93 Jul 6 '18 at 18:33
  • $\begingroup$ I edited my second comment in case, please take a look and thank you! $\endgroup$ – Haytam Jul 6 '18 at 18:36
  • $\begingroup$ When using tokens to code information (1 means something, 2 something else, etc) you need a more complex network that learns to distinguish the meaning of these codes, and learns that they do not represent order or quantity, yes. $\endgroup$ – BlueMoon93 Jul 6 '18 at 18:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.