Whenever you are doing reinforcement learning you periodically update the target network based on the weights of the Q network. While I do understand this helps create a stable target I do not understand how it helps the Q networks accuracy. If I am updating the Q network much more frequently than I am the target network then wont the Q network converge to and estimate of the target network that is inaccurate since the target network is being updated much slower ?

  • $\begingroup$ I think this ai.stackexchange.com/a/13459/69468 already answers your question $\endgroup$ Commented Jul 2, 2023 at 13:50
  • 1
    $\begingroup$ oh wow perfect this is exactly what I needed $\endgroup$
    – Stef
    Commented Jul 2, 2023 at 16:59