# Understanding the configuration of replay memory and epsilon in deep reinforcement learning

I am tentatively reusing a codebase of pacman to train my own deep reinforcement learning model. While most of the components seems reasonable and understandable to me, there are two things that seem obscure to me:

1. How to decide the size of the replay memory? Currently, since I set the total step of learning as 4000 (note that in the referred codebase this value is set as 4000000), so I just proportionally decrease the replay_memory_size as 400. Would that make sense?

2. What is the return value epsilon when calling function PiecewiseSchedule? I also proportionally decrease its parameters as follows:

        epsilon = PiecewiseSchedule([(0, 1.0),
(40, 1.0), # since we start training at 10000 steps
(80, 0.4),
(200, 0.2),
(400, 0.1),
(2000, 0.05)], outside_value=0.01)
replay_memory = PrioritizedReplayBuffer(replay_memory_size, replay_alpha)


where the original function call is like this:

        epsilon = PiecewiseSchedule([(0, 1.0),
(10000, 1.0), # since we start training at 10000 steps
(20000, 0.4),
(50000, 0.2),
(100000, 0.1),
(500000, 0.05)], outside_value=0.01)
replay_memory = PrioritizedReplayBuffer(replay_memory_size, replay_alpha)


And in general, what is the principle (guideline) behind setting a good size of "replay memory" and calling function PiecewiseSchedule? Thank you!

• Just to correct you, it is not reply memory, it is replay memory – Kartik Podugu May 30 '19 at 10:10
• my bad, thank you for the clarification – lllllllllllll May 30 '19 at 11:00