I'm trying to use DQN to solve the cart-pole environment. I have 2 networks (target and behavior). Both of them have 3 hidden layers with 24 neurons, using the ReLU activation. The loss is MSE and the optimizer is Adam. I copy the weights of the behavior network to the target network every 15 backpropagation steps.

My agent learns. Below you can see the total reward and running average plots.

enter image description here

However, it has a lot of "drops". Usually, after a couple of perfect sequences, it just "kills" the running average with a couple of very short episodes. What may be the reason for this behavior?



You must log in to answer this question.

Browse other questions tagged .