I have been dealing with a problem that I'm trying to solve with DQN. A general question that I have is regarding the target's update frequency. How should it change? Depending on what factor do we increase or decrease this hyperparameter?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
As you said yourself, it is a hyperparameter. Hence, no one (even you) can say what is the ideal update frequency. You have to test and try.
Having said that, remember one thing the target NN should mimic the actual network as closely as possible. Hence if you update it after a long number runs, then I think you will start losing the accuracy. On the contrary, if you update it too often, then you lose the benefit of using the target network (which is to boost the training rate and reduce training time) and the training will take a larger amount of time.
My suggestion is to try updating after every 5 to 8 episodes.
