2
$\begingroup$

Consider a 2D snake game, where the snake has to eat food to become longer. It must avoid hitting walls and biting into her tail.

Such a game could have a different amount of actions:

  • 3 actions: go straight, turn left, turn right (relative to crawling direction)
  • 4 actions: north, east, south, west (absolute direction on the 2D map)
  • 7 actions: a combination of option A and option B (leaves the preferred choice to the player)

While the game in principle is always the same, I would like to understand the impact of the amount of actions on the training of a neuronal network. One obvious thing is the number of output nodes of the neuronal network.

In case A (3 actions), the neuronal network cannot perform an incorrect action. Any of the 3 choices are valid moves.

In case B (4 actions), the net IMHO needs to learn that going into opposite direction does not have the desired effect and the snake continues moving into the old direction.

In case C (7 actions), the net needs to learn both, 1 action is always illegal and the 3 relative actions somehow map to the 3 absolute actions.

How can I consider the learning curve in these situations? Does option B need 25% more training than option A to achieve the same results (same fitness) (similar: option C needs 125% more training time)?

Is giving a negative reward for an impossible move considered cheating, because I do code the rules of the game into the reward logic?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.