There are several version of DDQN floating around. Sutton gives one that is a simple symmetric random update of the two Q functions. I think other papers (Silver paper for example) use a kind of delayed and split update rule.

Is there anything systematic describing the properties of the bias corrections and their respective advantages?

  • $\begingroup$ Double Q-learning was proposed by van Hasselt not Sutton neither Silver, there is an original Double Q-learning paper by van Hasselt that presents the algorithm for tabular case and there is also adaptation of that algorithm for function approximation case $\endgroup$
    – Brale
    Mar 14 '19 at 16:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.