# How does DQN convergence work in reinforcement learning

In supervised learning we have an unbiased target value, but in reinforcement learning this isn’t the case

The network predicts its own target value, now how exactly does it converge if the network predicts its target value

Can someone explain this to me ??

• Initially, the target will be totally wrong but as the process goes on and on it will become closer to the unbiased estimate, then it will eventually converge – Swakshar Deb Aug 7 '20 at 21:33
• Also, remember that this target is an unbiased estimate of the optimal Q value. – Swakshar Deb Aug 7 '20 at 21:40
• @Swakshar Deb Why does it get closer to the unbiased estimate, what exactly makes it that way – Chukwudi Ogbonna Aug 7 '20 at 22:32
• @Swakshar Deb why does it get closer to the unbiased estimate, what in that formula makes it the unbiased estimate – Chukwudi Ogbonna Aug 7 '20 at 22:32
• $q^{*}=E[R+\gamma \max_{a}Q(s_{t+1},a_{t+1})]$, that means $R+\gamma \max_{a} Q(s_{t+1},a_{t+1})$ is the unbiased estimate of $q^{*}$. So, if the iteration is large enough the target will eventually converge to the true estimate. – Swakshar Deb Aug 8 '20 at 6:25