Yes, the concept of dreaming or imagining has already been explored in reinforcement learning.
For example, have a look at Metacontrol for Adaptive Imagination-Based Optimization (2017) by Jessica B. Hamrick et al., which is a paper that I gave a talk/presentation on 1-2 years ago (though I don't remember well the details anymore).
There is also a blog post about the topic Agents that imagine and plan (2017) by DeepMind, which discusses two more recent papers and also mentions Hamrick's paper.
In 2018, another related and interesting paper was also presented at NIPS, i.e. World Models, by Ha and Schmidhuber.
If you search for "imagination/dreaming in reinforcement learning" on the web, you will find a lot more papers and articles about this interesting topic.