I have recently solved the Cartpole problem using double deep Q-learning. When I saw how the agent was doing, it used to go right every time, never left, and it did similar actions all the time.

Did the model overfit the environment? It seems that the agent just memorized the environment.

What are the common techniques to prevent the agent to overfit like that? Is that a common problem?

  • 1
    $\begingroup$ The concept of overfitting in reinforcement learning is not well-defined (see e.g. ai.stackexchange.com/q/20127/2444). In supervised learning, overfitting generally means that you learn the training data, but don't perform well on the test data. However, if you train your RL agent in the same exact environment where you want it to behave, you actually want it to "overfit", i.e. learn the environment dynamics. So, why don't you want the RL to learn the environment? Can you elaborate a little more? $\endgroup$ – nbro Jun 2 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.