2
$\begingroup$

During the training of DQN, I noticed that the model with prioritized experience replay (PER) had a smaller loss in general compared to a DQN without PER. The mean squared loss was an order of magnitude $10^{-5}$ for the DQN with PER, whereas the mean squared loss was an order of magnitude $10^{-2}$.

Do the smaller training errors have any effect on executing the final policy learned by the DQN?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I think it says something about the training progress, while another approach you can make sure is to look at the gradient norm. Sometimes, the training loss is really noisy while the gradient norm is much more clear.

Improve this answer
$\endgroup$
2
  • $\begingroup$ u mean the gradient of the weights of the model ? what would the gradient norm say about the training ? $\endgroup$
    – calveeen
    Commented May 12, 2020 at 0:51
  • $\begingroup$ well smaller norm would indicate that training is about to converge i guess. At a local minima, all the dimensions of a gradient would be 0. $\endgroup$
    – SpiderRico
    Commented Jun 11, 2020 at 0:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .