I am training a model using A2C with stable baselines 2. When I increased the timesteps I noticed that episode rewards seem to reset (see attached plot). I don´t understand where these sudden decays or resets could come from and I am looking for practical experience or pointers to theory what these resets could imply.
i've seen this in learning using deep Q network. some tips may help you came over this problem :
- use some remembering mechanisms like replay buffer. some times agent forget what it has been learnt. reply buffer remembers the agent what he saw at several episodes ago.
- something else that worked for me was changing the optimizer. As DQN article says , using RMSprop is very useful for learning neural network based agents.
if these tips dont help you, please give more information about your agent.