In Q-learning, during training, it doesn’t matter how I select actions. The algorithm always converges to optimal optimal policy. Why does this happen?

  • 1
    $\begingroup$ You can find a proof of the convergence of the Q-learning algorithm in the paper Convergence of Q-learning: A Simple Proof by Francisco S. Melo. $\endgroup$ – nbro Nov 18 '18 at 1:38
  • 1
    $\begingroup$ Your statement "it doesn’t matter how I select actions" is not really true. Q-learning "requires that all state-action pairs be visited infinitely often", as it's mentioned in the paper I linked you to above and e.g. in the book RL: An Introduction by Barto and Sutton. $\endgroup$ – nbro Nov 18 '18 at 1:40
  • 1
    $\begingroup$ So, what is your real question? Are you looking for a proof? If yes, then you can find it in the paper above. Or are you looking for an intuition behind the convergence of Q-learning? $\endgroup$ – nbro Nov 18 '18 at 1:41
  • $\begingroup$ I am actually looking for an intuition behind this. $\endgroup$ – Shifat E Arman Nov 20 '18 at 14:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.