# Problem over DQN Algorithm not converging on snake

I'm using a DQN Algorithm to play Snake.

The input of the neural network is a stack of 4 images taken from the games 80x80.

The output is an array of 4 values, one for every direction.

The problem is that the program does not converge and I've a lot of doubts in the replay function, where I train the neural network over a batch of 32 events.

That's the snippet:

def replay(self, batch_size):

minibatch = random.sample(self.memory, batch_size)

for state, action, reward, next_state, done in minibatch:

target = reward

if not done:
target = (reward + self.gamma *
np.amax(self.model.predict(next_state)[0]))
target_f = self.model.predict(state)
target_f[0][action] = target
self.model.fit(state, target_f, epochs=1, verbose=0)

if self.epsilon > self.epsilon_min:
self.epsilon *= self.epsilon_decay


Targets are:

• +1 for eating an apple
• 0 for doing a movement without dying
• -1000 for hitting a wall or the snake hitting himself
• Hi! questions about implementation are off-topic here, you may want to try other SE. Anyway, isn't -1000 a bit too much? – olinarr Jun 22 at 18:10
• Sorry. I was trying to convince the snake to not die but I can't get around the Q approximation with the neural network – Roberto Aureli Jun 22 at 18:15
• @olinarr Hi. You can vote to close this question or you still don't have such privilege? – nbro Jun 22 at 19:09
• @nbro flagged. Thanks for the reminder! – olinarr Jun 23 at 0:32

I think the main issue here is that you are trying to train the snake` (network) on images. This will create a lot of issues a there are no set parameters that the model can learn from.