I know we keep the target network constant during training to improve stability, but why exactly are we updating the weights of our target network? In particular, if we've already reached convergence, why exactly are we updating the weights of our target network?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
If you are certain that you reached convergence then there is no point in continuing to train your agent, because of that there is also no point in discussing why is target network being updated after convergence is reached. You should simply stop training if you converged. During training we obviously need to keep updating target network to improve correctness of Q-value estimates.
