I'm trying to solve the OpenAI's CarRacing-v0 environment with the DDPG algorithm. I've observed that after a period of learning, the agent's performance starts to deteriorate slowly. For some hyperparameter configurations this is followed by a rebound and again a slump. Here's what a typical reward plot looks like:
Since I'm new to reinforcement learning (this is my first shot at it), I don't know if this a common phenomenon. I know of catastrophic forgetting, but I believe that's not the case here, since this is more akin to a "languishing dementia". As far as I understand, "catastrophic forgetting" is an abrupt event, which contrasts with a gradual change I've been seeing in my attempts.
Is this some kind of general phenomenon with coverage in the existing literature or is this rather a quirk of my specific setup (algorithm + hyperparameters) for which the solution would be "change the setup"?
For reference, the implementation I'm using: https://github.com/hirekk/pytorch-rl