# In DQN, when do the parameters in the Neural Network update based on the reward received?

I'm aware that we back-propagate after computing the loss between:

The Neural Network Q values and the Target Network Q values

However, all this is doing is updating the parameters of the Neural Network to produce an output that matches the Target Q values as close as possible.

Suppose one epoch is run and the reward is +10, surely we need to update the parameters using this too to tell the Network to push the probability of these actions, given these parameters up.

How does the algorithm know +10 is good? Suppose the reward range is -10 for loss and +10 for win.

## 1 Answer

However, all this is doing is updating the parameters of the Neural Network to produce an output that matches the Target Q values as close as possible.

Yes. That is all it needs to do because we have defined the policy around the Q values like so:

$$\pi(s) = \text{argmax}_a \hat{q}(s,a,\theta)$$

Where $$\theta$$ is the neural network weights.

Therefore, if the estimates of Q are approximately the same as the action value of the optimal policy, the policy in DQN is approximately the optimal policy.

How does the algorithm know +10 is good?

It does not, at least not directly. The algorithm knows, approximately, what the action values are if it acts consistently with their current estimates by always choosing the maximising action at each step.

The learning process will learn that +10 is relatively good in your scenario because it never finds anything better when exploring.