2
$\begingroup$

Let's suppose that our RL agent needs to play a game with different levels. If we train our RL agent sequentially or with sequential data, our agent will learn how to play level 1, but then it will learn to play level 2 differently, because our agent learns how to play level 2 and forgets how to play level 1, since now our model is fitted using only experiences from level 2.

How does an experience replay buffer change this? Can you explain this in simple terms?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Random sampling from the replay memory helps us reach closer to the i.i.d assumption and also lets the learning algorithm train on the data multiple times compared to just once. Therefore, there is a lower chance of the network forgetting what it previously learned. However, the experience replay size is a hyperparameter and there are disadvantages of having it too low or too big. While the former disadvantage is self evident, the latter has been explained in this paper - A Deeper Look at Experience Replay

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .