I'm trying to train a network to navigate a 48x48 2D grid, and switch pixels from on to off or off to on. The agent receives a small reward if correct, and small punishment if incorrect pixel plotted.

I thought, like the Deepmind "Playing Atari with Deep Reinforcement Learning" Paper, I could just use only the pixel input, fed through 2 convolutional layers, to solve this task. The output of this is fed into 512 fully connected layer.

Unfortunately, it barely trains. When instead using additional vectors as input containing information about nearby pixels' state around the agent, the agent learns the task quite well (yet often orients the wrong awkwardly).

Each step, the agent moves up down left right, and plot pixel or not. The agent is visualized in the environemtn as a red square with white center dot. (also tried single red pixel). On-pixels within the red square are colored purple.

Is there something I can try to make the agent learn visual input better?

The orange line is the training with only visual observations, the grey one contained vector observations about the immediate neighboring pixel state as well.

agent's environmentTraining curves

  • 2
    $\begingroup$ Center image on the agent and fill the area outside of original image with black $\endgroup$ – mirror2image Jan 16 '20 at 12:40
  • $\begingroup$ @mirror2image interesting! I'll give that a try. What would be a reason for centering making the a better representation? $\endgroup$ – SumakuTension Jan 16 '20 at 13:18
  • 1
    $\begingroup$ Agent movement will have much bigger effect on environment and action becoming more pronounced - it more easy to detect movement of whole image then movement of small part of it. $\endgroup$ – mirror2image Jan 16 '20 at 14:40
  • $\begingroup$ @mirror2image that makes sense ya. Will give it a go! $\endgroup$ – SumakuTension Jan 16 '20 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.