# What could be the cause of the drop in the reward in A3C?

The mean episodic reward is generally increasing, but it has spontaneous drops, and I'm not sure of their cause.

The problem has a sparse reward, batch size=2000, entropy_coefficient=0.1, other hyper-parameters are pretty standard.

Has anyone seen this kind of behavior? What could be the cause these drops in the reward(not enough exploration, too sparse rewards, the state not expressive enough, etc.)?