My agent uses an $\epsilon$-greedy strategy to learn. The exploration rate (i.e. $\epsilon$) decays throughout the training. I've seen examples where people update $\epsilon$ every time an action is taken, while others update it at the end of the episode. If updated at every action, $\epsilon$ is more continuous. Does it matter? Is there a standard? Is one better than another?

  • $\begingroup$ In the long run it would make no difference and both have their benefits. Updating after an episode will take longer to converge but offers more exploration and vice-versa. $\endgroup$ – David Ireland Oct 30 '20 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.