I'm trying to use DQN to solve the cart-pole environment. I have 2 networks (target and behavior). Both of them have 3 hidden layers with 24 neurons, using the ReLU activation. The loss is MSE and the optimizer is Adam. I copy the weights of the behavior network to the target network every 15 backpropagation steps.

My agent learns. Below you can see the total reward and running average plots.

enter image description here

However, it has a lot of "drops". Usually, after a couple of perfect sequences, it just "kills" the running average with a couple of very short episodes. What may be the reason for this behavior?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.