8
$\begingroup$

Drawing parallels between Machine Learning techniques and a human brain is a dangerous operation. When it is done successfully, it can be a powerful tool for vulgarisation, but when it is done with no precaution, it can lead to major misunderstandings.

I was recently attending a conference where the speaker described Experience Replay in RL as a way of making the net "dream". I'm wondering how true this assertion is. The speaker argued that a dream is a random addition of memories, just as experience replay. However, I doubt the brain remembers its dream or either learns from it. What is your analysis?

$\endgroup$
0
8
$\begingroup$

The speaker argued that a dream is a random addition of memories, just as experience replay.

The speaker is taking some liberties due to a general lack of scientific understanding of what dreams are. We don't even have strong consensus on why sleep is a necessary feature of animals, let alone what part dreaming plays in it. However, there are some widely-accepted theories, with supporting evidence, that dreams are part of a learning and memorisation process. Studies that manipulate sleep or dreaming have shown changes in the speed that skills are learned for example.

Experience replay in reinforcement learning is a far more precise and well-understood affair, whereby individual time steps that occurred in the past are visited and re-assessed in light of current knowledge about long-term value, at random. If dreams were really like experience replay as it is practiced in RL today, then they would consist of a random jumble of tiny seemingly inconsequential events strung together, and all taken very exactly from the events of the past day. Sometimes dreams do contain content like this, but typically the content is far more varied.

Taken with a large dose of artistic license, then yes, the speaker is referring to real theories and conjectures about dreaming, that do have scientific support. Although it is equally good to draw parallels between dreams and a higher-level management of the memory or experience replay data - which items to replay, and which to keep, depending on what is salient about the information. For instance, there is good evidence that dreams help filter what is forgotten, and also evidence that events associated with strong emotional state are more likely to feature in dreams.

It is important to separate the speaker's analogy, and any suggestion that a current reinforcement learning agent has a subjective experience. We are still a long way away from anything like that, and other similar use of a dreaming metaphor in machine learning - e.g. "Deep Dream" - is equally not an assertion that the devices are having an experience of any kind.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.