1
$\begingroup$

enter image description here

The agent is trying to master the Atari Breakout game.

Here is my code

Is that normal that reward_100 decreased that much after it hits 4.5? Is there a way to avoid that behavior?

Be aware that reward_100 is simply mean_reward = np.mean(self.total_rewards[-100:]). In other words, it is the mean over the last 100 rewards. On the graph, reward_100 represents de y-axis and th number of episodes the x-axis.

$\endgroup$
6
  • $\begingroup$ These are just general questions about your ML approach. Are you decaying the learning rate? And how are you trading off exploitation and exploration? $\endgroup$ – nbro Apr 14 '20 at 20:51
  • $\begingroup$ Decaying the learning rate seems to be the right thing to do. It changed a lot of things. $\endgroup$ – jgauth Apr 14 '20 at 21:48
  • 2
    $\begingroup$ Nice! Maybe you can provide a formal answer below, once you solve your problem ;) $\endgroup$ – nbro Apr 14 '20 at 22:00
  • $\begingroup$ What algorithm are you using? $\endgroup$ – Ray Walker Apr 15 '20 at 15:13
  • $\begingroup$ @RayWalker I used just a standard DQN. Look at my code in the question to know more info. $\endgroup$ – jgauth Apr 15 '20 at 15:17
1
$\begingroup$

It seems that decaying the learning rate solved my problem. I changed learning_rate from 0.001 to 0.0001

enter image description here

$\endgroup$
2
  • $\begingroup$ Hi jgauth, great that decreasing the learning rate has solved your issue. By "decaying the learning rate" @nbro most probably meant to decrease the learning rate during training. This means that instead of a fixed learning rate you use a function that takes the amount of passed training steps as input and outputs a learning rate. $\endgroup$ – Ray Walker Apr 15 '20 at 15:22
  • $\begingroup$ @RayWalker Thanks, I did not notice that. It might be a good idea! $\endgroup$ – jgauth Apr 15 '20 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.