I'm trying to replicate the results of the DeepMind's paper with Breakout included in OpenAI Gym. I wonder how much frames should I keep until I reach the fixed exploration rate. Actually it reaches the minimum value at about 1,5M frames (about 5000 episodes). It is much? If it is, does it impact negatively on training? My minimum rate is fixed at 0.01.