Consider a 2D snake game, where the snake has to eat food to become longer. It must avoid hitting walls and biting into her tail.
Such a game could have a different amount of actions:
- 3 actions: go straight, turn left, turn right (relative to crawling direction)
- 4 actions: north, east, south, west (absolute direction on the 2D map)
- 7 actions: a combination of option A and option B (leaves the preferred choice to the player)
While the game in principle is always the same, I would like to understand the impact of the amount of actions on the training of a neuronal network. One obvious thing is the number of output nodes of the neuronal network.
In case A (3 actions), the neuronal network cannot perform an incorrect action. Any of the 3 choices are valid moves.
In case B (4 actions), the net IMHO needs to learn that going into opposite direction does not have the desired effect and the snake continues moving into the old direction.
In case C (7 actions), the net needs to learn both, 1 action is always illegal and the 3 relative actions somehow map to the 3 absolute actions.
How can I consider the learning curve in these situations? Does option B need 25% more training than option A to achieve the same results (same fitness) (similar: option C needs 125% more training time)?
Is giving a negative reward for an impossible move considered cheating, because I do code the rules of the game into the reward logic?