In supervised learning we have an unbiased target value, but in reinforcement learning this isn’t the case

The network predicts its own target value, now how exactly does it converge if the network predicts its target value

Can someone explain this to me ??

enter image description here

  • $\begingroup$ Initially, the target will be totally wrong but as the process goes on and on it will become closer to the unbiased estimate, then it will eventually converge $\endgroup$ Aug 7 '20 at 21:33
  • $\begingroup$ Also, remember that this target is an unbiased estimate of the optimal Q value. $\endgroup$ Aug 7 '20 at 21:40
  • $\begingroup$ @Swakshar Deb Why does it get closer to the unbiased estimate, what exactly makes it that way $\endgroup$ Aug 7 '20 at 22:32
  • $\begingroup$ @Swakshar Deb why does it get closer to the unbiased estimate, what in that formula makes it the unbiased estimate $\endgroup$ Aug 7 '20 at 22:32
  • $\begingroup$ $q^{*}=E[R+\gamma \max_{a}Q(s_{t+1},a_{t+1})]$, that means $R+\gamma \max_{a} Q(s_{t+1},a_{t+1})$ is the unbiased estimate of $q^{*}$. So, if the iteration is large enough the target will eventually converge to the true estimate. $\endgroup$ Aug 8 '20 at 6:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.