5
$\begingroup$

Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution?

For example:

  • I use level 1 experiences, my model $p$ is fitted to learn how to play that level.

  • I go to level 2, my weights are updated and fitted to play level 2 meaning I don't know how to play level 1 again.

$\endgroup$
2
  • 1
    $\begingroup$ Please, do not ask the same question in different posts only because you have not yet received an answer. You already asked about ER here: ai.stackexchange.com/q/22694/2444. I am sure someone will answer that question. $\endgroup$
    – nbro
    Jul 27, 2020 at 13:17
  • $\begingroup$ This question is very related to ai.stackexchange.com/q/13289/2444, although I wouldn't say it's a duplicate because yours is specific to DQN. $\endgroup$
    – nbro
    Jul 28, 2020 at 13:10

1 Answer 1

5
$\begingroup$

You are referring to catastrophic forgetting which could be an issue in any neural net. More specifically for DQN refer to this article.

$\endgroup$
3
  • 1
    $\begingroup$ This was incredibly helpful, Thank you $\endgroup$
    – Chukwudi
    Jul 27, 2020 at 12:20
  • 1
    $\begingroup$ I also have a question, the issue is our replay memory size can’t be too large because of performance issues, too small and it’s irrelevant, so if we have a large space state with multiple tasks, even replay memory wouldn’t be able to help with the catastrophic forgetting, so what can be the solution? $\endgroup$
    – Chukwudi
    Jul 27, 2020 at 12:23
  • $\begingroup$ @Chukwudi I'm not really sure, sorry. $\endgroup$
    – pedrum
    Jul 27, 2020 at 13:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .