How should I decay the $\epsilon$ in Q-learning?
Currently, I am decaying epsilon as follows. I initialize $\epsilon$ to be 1, then, after every episode, I multiply it by some $C$ (let it be $0.999$), when it reaches $0.01$. After that, I keep $\epsilon$ to be $0.01$ all the time. I think this has a terrible consequence.
So, I need a $\epsilon$ decay algorithm. I haven't found script or formula about it, so can you tell me?