Drawing parallels between Machine Learning techniques and a human brain is a dangerous operation. When it is done successfully, it can be a powerful tool for vulgarisation, but when it is done with no precaution, it can lead to major misunderstandings.

I was recently attending a conference where the speaker described Experience Replay in RL a way of making the net "dream". I'm wondering how true this assertion is. The speaker argued that a dream is a random addition of memories, just as experience replay. However, I doubt the brain remembers its dream or either learns from it. What is your analysis?

  • $\begingroup$ I would argue all brain related stuff has 2 parts: psychological and the biophysical and mathematical part. Dream falls in the psychological part of brain studies. Machine Learning on the other hand is a mathematical subject. So comparing mathematical things to psychological things is a waste of time according to me. A Neural network may simulate a dream, but it will not understand it nor will anything useful come out of it except maybe some cool mobile applications like Neural style transfer I guess. $\endgroup$ – DuttaA Sep 9 '18 at 19:23

The speaker argued that a dream is a random addition of memories, just as experience replay.

The speaker is taking some liberties due to a general lack of scientific understanding of what dreams are. We don't even have strong consensus on why sleep is a necessary feature of animals, let alone what part dreaming plays in it. However, there are some widely-accepted theories, with supporting evidence, that dreams are part of a learning and memorisation process. Studies that manipulate sleep or dreaming have shown changes in the speed that skills are learned for example.

Experience replay in reinforcement learning is a far more precise and well-understood affair, whereby individual time steps that occurred in the past are visited and re-assessed in light of current knowledge about long-term value, at random. If dreams were really like experience replay as it is practiced in RL today, then they would consist of a random jumble of tiny seemingly inconsequential events strung together, and all taken very exactly from the events of the past day. Sometimes dreams do contain content like this, but typically the content is far more varied.

Taken with a large dose of artistic license, then yes, the speaker is referring to real theories and conjectures about dreaming, that do have scientific support. Although it is equally good to draw parallels between dreams and a higher-level management of the memory or experience replay data - which items to replay, and which to keep, depending on what is salient about the information. For instance, there is good evidence that dreams help filter what is forgotten, and also evidence that events associated with strong emotional state are more likely to feature in dreams.

It is important to separate the speaker's analogy, and any suggestion that a current reinforcement learning agent has a subjective experience. We are still a long way away from anything like that, and other similar use of a dreaming metaphor in machine learning - e.g. "Deep Dream" - is equally not an assertion that the devices are having an experience of any kind.

| improve this answer | |
  • $\begingroup$ Quite correct I think...since dreams are not understood...especially converting it to formal proof/logic. $\endgroup$ – DuttaA Sep 10 '18 at 11:34
  • $\begingroup$ Thanks for your developped answer ! The links are also interesting :). $\endgroup$ – 16Aghnar Sep 11 '18 at 10:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.