3
$\begingroup$

In order to learn about DP and RL, I chose to start a side project where I would train an AI to play a "simple" card game. I will be doing this using the DQN with replay memory.
The problem is, I can't get the intuition behind how to represent the input to the neural network..

About the game

It's a fairly simple 2-players game. There is a deck of 40 unique cards (4 types of cards, 10 numbered cards in each type).
Each player gets 4 cards and each turn a player must put a card on the table.
If a player puts a card and there is already a card with the same number on the table, the player wins both cards.
If for example a player plays Card 2 and on the table there is Cards 2, 3, 4, 5 then the player wins all those cards (sequence).
Cards won don't go back to the hand nor to the deck, they are just kept as like a score.
When the players have 0 cards in hand, another 4 cards are dealt to each one untile the deck has 0 cards left where then we decide who won based on the number of cards eaten/won.

Question

As the input, I will be using the following:

  • Current cards in the AI hand (40 one-hot-encoded features?)
  • Current cards on the table (40 one-hot-encoded features?)
  • History of played cards (40 one-hot-encoded features?)

This would give 120 columns/features in each state.
I am wondering wheter this is too much for the NN or wheter my input representation would be bad for the NN?
Should the features be represented as a (120,) vector or as a 3x40 matrix?

I am also wondering if it's a good idea to represent the current cards on the table as just a 10 one-hot-encoded features since the type of the cards don't matter and the same number can't exist 2 times in the table?

Thank you in advance.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

120 inputs can be handled by a complex enough network. Dealing with high complexity is one of NN's strengths.

Using a (120,) vector or a (3,40) matrix is the same, they're still 120 inputs. Your binary encoding should work. Another option is a single (40,) vector, with 0 being "still in deck", 1 being "in hand", 2 being "on table", 3 being "already played".

If the types of cards are irrelevant, you could actually have a (3,10) matrix with counters of cards (1 in hand, 1 in table, 2 already played). You can try different approaches and see what works best.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Never knew that we can encode features as non binary... Can you tell me the name of this "technique"? Thank you for the informations. $\endgroup$
    – Haytam
    Jul 6, 2018 at 18:12
  • $\begingroup$ For the (40,) vector, wouldn't encoding like that mean that there is some kind of order? $\endgroup$
    – Haytam
    Jul 6, 2018 at 18:30
  • $\begingroup$ Imagine you had one ace in hand, one on table, 2 played already. How could you code that with a single (10,) vector? You need 3 rows, one for each place. Its simply called non binary input, its very common. Image processing NNs use inputs with pixel values [0,255] $\endgroup$
    – BlueMoon93
    Jul 6, 2018 at 18:33
  • $\begingroup$ I edited my second comment in case, please take a look and thank you! $\endgroup$
    – Haytam
    Jul 6, 2018 at 18:36
  • $\begingroup$ When using tokens to code information (1 means something, 2 something else, etc) you need a more complex network that learns to distinguish the meaning of these codes, and learns that they do not represent order or quantity, yes. $\endgroup$
    – BlueMoon93
    Jul 6, 2018 at 18:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .