4
$\begingroup$

Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution?

For example:

  • I use level 1 experiences, my model $p$ is fitted to learn how to play that level.

  • I go to level 2, my weights are updated and fitted to play level 2 meaning I don't know how to play level 1 again.

$\endgroup$
  • 1
    $\begingroup$ Please, do not ask the same question in different posts only because you have not yet received an answer. You already asked about ER here: ai.stackexchange.com/q/22694/2444. I am sure someone will answer that question. $\endgroup$ – nbro Jul 27 at 13:17
  • $\begingroup$ This question is very related to ai.stackexchange.com/q/13289/2444, although I wouldn't say it's a duplicate because yours is specific to DQN. $\endgroup$ – nbro Jul 28 at 13:10
3
$\begingroup$

You are referring to catastrophic forgetting which could be an issue in any neural net. More specifically for DQN refer to this article.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ This was incredibly helpful, Thank you $\endgroup$ – Chukwudi Jul 27 at 12:20
  • 1
    $\begingroup$ I also have a question, the issue is our replay memory size can’t be too large because of performance issues, too small and it’s irrelevant, so if we have a large space state with multiple tasks, even replay memory wouldn’t be able to help with the catastrophic forgetting, so what can be the solution? $\endgroup$ – Chukwudi Jul 27 at 12:23
  • $\begingroup$ @Chukwudi I'm not really sure, sorry. $\endgroup$ – pedrum Jul 27 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.