-2
$\begingroup$

I'm using a DQN Algorithm to play Snake.

The input of the neural network is a stack of 4 images taken from the games 80x80.

The output is an array of 4 values, one for every direction.

The problem is that the program does not converge and I've a lot of doubts in the replay function, where I train the neural network over a batch of 32 events.

That's the snippet:

def replay(self, batch_size):

    minibatch = random.sample(self.memory, batch_size)

    for state, action, reward, next_state, done in minibatch:

        target = reward

        if not done:
            target = (reward + self.gamma *
                      np.amax(self.model.predict(next_state)[0]))
        target_f = self.model.predict(state)
        target_f[0][action] = target
        self.model.fit(state, target_f, epochs=1, verbose=0)

    if self.epsilon > self.epsilon_min:
        self.epsilon *= self.epsilon_decay`

Targets are:

  • +1 for eating an apple
  • 0 for doing a movement without dying
  • -1000 for hitting a wall or the snake hitting himself
$\endgroup$
  • $\begingroup$ Hi! questions about implementation are off-topic here, you may want to try other SE. Anyway, isn't -1000 a bit too much? $\endgroup$ – olinarr Jun 22 at 18:10
  • $\begingroup$ Sorry. I was trying to convince the snake to not die but I can't get around the Q approximation with the neural network $\endgroup$ – Roberto Aureli Jun 22 at 18:15
  • 1
    $\begingroup$ @olinarr Hi. You can vote to close this question or you still don't have such privilege? $\endgroup$ – nbro Jun 22 at 19:09
  • $\begingroup$ @nbro flagged. Thanks for the reminder! $\endgroup$ – olinarr Jun 23 at 0:32
0
$\begingroup$

I think the main issue here is that you are trying to train the snake (network) on images. This will create a lot of issues a there are no set parameters that the model can learn from.

From images, there is no logical way to define the boundary, directions and objects on the board. It will be much easier to write a simple computer vision script of game API to provide actual meaningful inputs to the model.

Here is a great article on building a model to play the snake game. The author also provides the game API for input along with example code to train the snake game.

Final results from the model

Snake agent results

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.