I recently tried to reproduce the results of double Q-learning. However, the results are not satisfying. I have also tried to compare double Q learning with Q-learning in Taxi-v3, FrozenLake without slippery, Roulette-v0, etc. But Q-learning outperforms double Q-learning in all of these environments.

I am not sure whether if there is something wrong with my implementation as many materials about double Q actually focus on double DQN. While at the same time of checking, I wonder is there any toy example that can exemplify the performance of double Q-learning?

  • 1
    $\begingroup$ Have you used the same parameters and hyper-parameters as the ones in the paper for the roulette environment? $\endgroup$
    – nbro
    Dec 23 '20 at 9:31
  • $\begingroup$ @nbro Thanks for your comment. I haven't tested that in the paper's roulette as I am not very understand the environment settings. Consider giving it a try another day. $\endgroup$
    – Allen_FrCh
    Dec 23 '20 at 11:38
  • $\begingroup$ have you seen the example from the Sutton and Barto book? $\endgroup$ Dec 26 '20 at 23:05
  • $\begingroup$ @DavidIreland Haven't yet. Thanks for suggestion. $\endgroup$
    – Allen_FrCh
    Dec 27 '20 at 7:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.