Consider a 2D snake game, where the snake has to eat food to become longer. It must avoid hitting walls and biting into her tail.

Such a game could have a different amount of actions:

  • 3 actions: go straight, turn left, turn right (relative to crawling direction)
  • 4 actions: north, east, south, west (absolute direction on the 2D map)
  • 7 actions: a combination of option A and option B (leaves the preferred choice to the player)

While the game in principle is always the same, I would like to understand the impact of the amount of actions on the training of a neuronal network. One obvious thing is the number of output nodes of the neuronal network.

In case A (3 actions), the neuronal network cannot perform an incorrect action. Any of the 3 choices are valid moves.

In case B (4 actions), the net IMHO needs to learn that going into opposite direction does not have the desired effect and the snake continues moving into the old direction.

In case C (7 actions), the net needs to learn both, 1 action is always illegal and the 3 relative actions somehow map to the 3 absolute actions.

How can I consider the learning curve in these situations? Does option B need 25% more training than option A to achieve the same results (same fitness) (similar: option C needs 125% more training time)?

Is giving a negative reward for an impossible move considered cheating, because I do code the rules of the game into the reward logic?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.