1
$\begingroup$

I've found online this interesting algorithm: r

From what I understand reading this algorithm, I can't figure out why I should "perform the opposite action" and consequently storing that second experience as then it is never used to update or doing anything else. Is this algorithm incorrect?

$\endgroup$
  • $\begingroup$ It's used, they store both observations in $M$ and then they sample from it, just because they use notation $e_i = (o_i,a_i,r_i,o_{i+1})$ it doesn't mean they sample only observations from $\mathcal E$. $\endgroup$ – Brale May 28 at 8:50
  • $\begingroup$ @Brale You're right, thanks! $\endgroup$ – aandre_90 May 28 at 9:01
  • $\begingroup$ @Brale just another question on this algorithm: when computing $y_i$ they use the argmax operator but w.r.t the main DNN. I've seen other pseudo algorithm which include the target network that compute the action w.r.t the target netowrk. What is the difference? $\endgroup$ – aandre_90 Jun 3 at 7:42
  • 1
    $\begingroup$ This is some variation of double Q learning, look up Deep Double Q Learning $\endgroup$ – Brale Jun 3 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.