I currently implemented Deep Neuroevolution and used it on a couple of Atari games. For my implementation I used a similar Genetic Algorithm, network and setup as the Uber AI Deep Neuroevolution paper from 2018. Unfortunately my policies are not "diverse" and always take the same actions. So for instance I have one policy that always takes action 1 and another that always takes action 2. After many mutations, the policies change but still always take the same action i.e. I now have policies that only take move 3. Even after thousands of games and mutations they don't seem to improve. Particularly this effect occurs for the game of Pong. I believe this might be due to the fact that for Pong only few pixels change during the game (the player and the ball), which might lead to Neuroevolution evolving networks that don't pick up these small changes. Any idea how one can fix this?