This is a q-learning snake using a neural network as a q function aproximator and I'm losing my mind here the current model it's worst than the initial one.
The current model uses a 32x32x32 MLPRegressor from scikit-learn using relu as activation function and the adam solver.
The reward function is like following:
- death reward = -100.0
- alive reward = -10.0
- apple reward = 100.0
The features extracted from each state are the following:
- what is in front of the snake's head(apple, empty, snake)
- what is in the left of the snake's head
- what is in the right of the snake's head
- euclidian distance between head and apple
- the direction from head to the apple measured in radians
- length of the snake
One episode consists of the snake playing until it dies, I'm also using in training a probability epsilon that represent the probability that the snake will take a random action if this isn't satisfied the snake will take the action for which the neural network gives the biggest score, this epsilon probability gradually decrements after each iteration.
The episode is learned by the regressor in reverse order one statet-action at a time.
However the neural network fails too aproximate the q function, no matter how many iterations the snake takes the same action for any state.
Things I tried:
- changing the structure of the neural network
- changing the reward function
- changing the features extracted, I even tried passing the whole map to the network
Code (python): https://pastebin.com/57qLbjQZ