1
$\begingroup$

enter image description here

The agent is trying to master the Atari Breakout game.

Here is my code

Is that normal that reward_100 decreased that much after it hits 4.5? Is there a way to avoid that behavior?

Be aware that reward_100 is simply mean_reward = np.mean(self.total_rewards[-100:]). In other words, it is the mean over the last 100 rewards. On the graph, reward_100 represents de y-axis and th number of episodes the x-axis.

Improve this question
$\endgroup$
6
  • $\begingroup$ These are just general questions about your ML approach. Are you decaying the learning rate? And how are you trading off exploitation and exploration? $\endgroup$
    – nbro
    Apr 14, 2020 at 20:51
  • $\begingroup$ Decaying the learning rate seems to be the right thing to do. It changed a lot of things. $\endgroup$
    – jgauth
    Apr 14, 2020 at 21:48
  • 2
    $\begingroup$ Nice! Maybe you can provide a formal answer below, once you solve your problem ;) $\endgroup$
    – nbro
    Apr 14, 2020 at 22:00
  • $\begingroup$ What algorithm are you using? $\endgroup$
    – Ray Walker
    Apr 15, 2020 at 15:13
  • $\begingroup$ @RayWalker I used just a standard DQN. Look at my code in the question to know more info. $\endgroup$
    – jgauth
    Apr 15, 2020 at 15:17

1 Answer 1

Reset to default
1
$\begingroup$

It seems that decaying the learning rate solved my problem. I changed learning_rate from 0.001 to 0.0001

enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi jgauth, great that decreasing the learning rate has solved your issue. By "decaying the learning rate" @nbro most probably meant to decrease the learning rate during training. This means that instead of a fixed learning rate you use a function that takes the amount of passed training steps as input and outputs a learning rate. $\endgroup$
    – Ray Walker
    Apr 15, 2020 at 15:22
  • $\begingroup$ @RayWalker Thanks, I did not notice that. It might be a good idea! $\endgroup$
    – jgauth
    Apr 15, 2020 at 16:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .