I'm trying to train a network to navigate a 48x48 2D grid, and switch pixels from on to off or off to on. The agent receives a small reward if correct, and small punishment if incorrect pixel plotted.
I thought, like the Deepmind "Playing Atari with Deep Reinforcement Learning" Paper, I could just use only the pixel input, fed through 2 convolutional layers, to solve this task. The output of this is fed into 512 fully connected layer.
Unfortunately, it barely trains. When instead using additional vectors as input containing information about nearby pixels' state around the agent, the agent learns the task quite well (yet often orients the wrong awkwardly).
Each step, the agent moves up down left right, and plot pixel or not. The agent is visualized in the environemtn as a red square with white center dot. (also tried single red pixel). On-pixels within the red square are colored purple.
Is there something I can try to make the agent learn visual input better?
The orange line is the training with only visual observations, the grey one contained vector observations about the immediate neighboring pixel state as well.