2
$\begingroup$

I've read most of the posts on here regarding this subject, however most of them deal with gameboards where there are two different categories of single pieces on a board without walls etc.

My game board has walls, and multiple instances of food. There are 8 different categories, Walls, enemy food, my food, enemy powerup, my powerup, attackable enemies, threatening enemies, and current teammate.

I have one hot encoded all of this data into a tensor of size (8, 16, 32) where (16, 32) are the sizes of the game grid. However I'm not sure whether this is appropriate since many of the categories have multiple occurrences of each category in a single (walls, food). Is it appropriate to use one hot encoding to represent categories in spatial data, where multiple one's may be present?

The alternative I was considering was to use a CNN, however many posts have said it is inappropriate for one hot data. My reasoning was that since the data is a abstract Boolean grid representing the RGB frames, it might be appropriate.

Does anyone have any suggestions as to the best way to represent a spatial Boolean grid representing multiple categories for input to a network?

$\endgroup$

2 Answers 2

1
$\begingroup$

The way you describe with one hot encoding is correct.

Note that how the state is encoded is a separate question from the neural network, so I'm not sure what convolutional neural networks have to do with the question. In the famous atari game example, the input is a sequence of RGB images; a cnn is used to process the images. In your example you probably just want to use a regular Dense network, as your input is just the one hot encoding and not images.

$\endgroup$
2
  • $\begingroup$ Ahh, i've gone with a Dense network now. Do you think I should use activation functions? at the moment my network is outputting values between -1 and 1. I'm guessing it'd be better to use the probabilities as my reward function is a sum of percentages $\endgroup$ Sep 7, 2021 at 2:28
  • $\begingroup$ Q tables/functions are attempting to predict the rewards for a given action assuming all of the perfect decisions are made from there forward until the end of the episode. It is not only the present reward but a discounted value of all future rewards. So no matter the value of the rewards, an output neuron for the Q-Value of an action should be able to represent (-inf, inf), this is typically linear activation. $\endgroup$
    – foreverska
    Jan 22 at 16:53
1
$\begingroup$

If there can be multiple items on a square something more complex than one-hot is needed. As the name implies, only one category can be hot (set to true) in one-hot. You need multi-hot. Further, if multiple instances of items can be on a grid square (2 food, 1 wall) it might be beneficial to encode it proportionality-wise (.66 food, .33 wall).

CNN's inductive bias is locality. It's looking for items near each other in the grid space with certain values. It identifies groups of items that are similar to some pattern (ie looking for the ball or paddle in pong). I'll leave it to you to decide if this is interesting in your environment.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .