I am trying to build a DQN model for the Atari Pong game, but I am not sure whether the model is learning at all.

I am using the architecture described in the paper Playing Atari with Deep Reinforcement Learning. And I tested the model on a simpler environment (like CartPole), which worked great, but I am not seeing any progress at all with Pong, I have been training the model for 2-3 hours and its performance is no better than taking random actions.

Should I just keep waiting or there might be something wrong with my code. Around how many episodes should it take before I see some positive results?

  • 1
    $\begingroup$ Don't bother with hours, everyone has different hardware. They say in paper that they train for 10 million frames which is 10 million timesteps in the environment. How many did you complete in 2-3 hours ? $\endgroup$ – Brale Apr 1 '20 at 13:56
  • $\begingroup$ @Brale_ 100 episodes takes roughly 2 hours, and each episode takes maximum 10,000 steps (I am running each episode until game is lost/won). $\endgroup$ – Ach113 Apr 1 '20 at 14:11
  • $\begingroup$ I don't expect same results as in paper in just 100 episodes, but I should be getting at least some improvements right? $\endgroup$ – Ach113 Apr 1 '20 at 14:32
  • 1
    $\begingroup$ Well for you it could take entire day to reach around 10 million timesteps. Leave it over night for 7-8 hours and see in the morning if it gets any better. If not then you might have some bug. $\endgroup$ – Brale Apr 1 '20 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.