I'm trying to train a network to navigate a 48x48 2D grid, and switch pixels from on to off or off to on. The agent receives a small reward if correct, and small punishment if incorrect pixel plotted.

I thought, like the Deepmind "Playing Atari with Deep Reinforcement Learning" Paper, I could just use only the pixel input, fed through 2 convolutional layers, to solve this task. The output of this is fed into 512 fully connected layer.

Unfortunately, it barely trains. When instead using additional vectors as input containing information about nearby pixels' state around the agent, the agent learns the task quite well (yet often orients the wrong awkwardly).

Each step, the agent moves up down left right, and plot pixel or not. The agent is visualized in the environemtn as a red square with white center dot. (also tried single red pixel). On-pixels within the red square are colored purple.

Is there something I can try to make the agent learn visual input better?

The orange line is the training with only visual observations, the grey one contained vector observations about the immediate neighboring pixel state as well.

agent's environmentTraining curves

  • 2
    $\begingroup$ Center image on the agent and fill the area outside of original image with black $\endgroup$ Jan 16, 2020 at 12:40
  • $\begingroup$ @mirror2image interesting! I'll give that a try. What would be a reason for centering making the a better representation? $\endgroup$ Jan 16, 2020 at 13:18
  • 1
    $\begingroup$ Agent movement will have much bigger effect on environment and action becoming more pronounced - it more easy to detect movement of whole image then movement of small part of it. $\endgroup$ Jan 16, 2020 at 14:40
  • $\begingroup$ @mirror2image that makes sense ya. Will give it a go! $\endgroup$ Jan 16, 2020 at 14:57


You must log in to answer this question.