7
$\begingroup$

While reading the DQN paper, I found that randomly selecting and learning samples reduced divergence in RL using a non-linear function approximator (e.g a neural network).

So why does Reinforcement Learning using a non-linear function approximator diverge when using strongly correlated data as input?

$\endgroup$
  • $\begingroup$ I apologize the question was not clear. Let me rephrase and elaborate the question again. I was wondering why Reinforcement Learning using a non-linear function approximator diverge when using strongly correlated data as input? $\endgroup$ – 강문주 Feb 11 at 6:11
  • $\begingroup$ read chapter 11 of this book. This is only a draft, if you can find full book even better. Also, I think similar questions were answered already so try searching a bit through the website $\endgroup$ – Brale Feb 11 at 8:19
  • $\begingroup$ I had asked a similar but more general question a few months ago Why doesn't Q-learning converge when using function approximation?, but your question is more specific, so I belive they are not duplicates. $\endgroup$ – nbro Feb 11 at 16:05
  • $\begingroup$ You write about a paper.... could you give us a link, if available? $\endgroup$ – maaartinus Feb 11 at 23:40
  • $\begingroup$ this is link about DQN paper that i read. arxiv.org/abs/1312.5602. $\endgroup$ – 강문주 Feb 12 at 2:40
2
$\begingroup$

It is not so much the problem of using Reinforcement Learning to train the neural networks, it is the assumptions made about the data given to standard Neural Networks. They are not capable of handling strongly correlated data which is one of the motivations for introducing Recurrent Neural Networks, as they can handle this correlated data well.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.