# Reward and loss follow the same shape in DQN

If the accumulated reward increases, the loss increases and vice versa. This is a strange behaviour. See the figure below for an example. What is the possibility of having this behaviour in DQN?

I have $$3000$$ episodes. An episode lasts $$20$$ time steps. So, in total, I have $$60000$$ time steps. I am using the ADAM optimizer with a learning rate of $$5\times10^{-6}$$. The target update frequency is once every $$2000$$ time steps (once every $$100$$ episodes). The training happens every episode (once every $$20$$ time steps). I am using double DQN with the duelling architecture and the prioritized experience replay buffer. The size of the buffer is $$300$$ and the batch size is $$128$$.

I tried to modify some hyperparameters but for now I still have this strange behaviour.

• What is your loss function? Also, your replay buffer is way too small. Can you train it with a rp buffer size of ~10,000 and check if the error persists? – Leon Shams Sep 14 at 8:12