# Sample minibatch from the memory minibatch = random.sample(self.memory, batch_size) # Extract informations from each memory for state, action, reward, next_state, done in minibatch: # if done, make our target reward target = reward if not done: # predict the future discounted reward target = reward + self.gamma * \ np.amax(self.model.predict(next_state)) # make the agent to approximately map # the current state to future discounted reward # We'll call that target_f target_f = self.model.predict(state) target_f[action] = target
Why can't they use mini batches instead? I'm new to RL, but in deep learning people tends to use mini-batches as they would result in a more stable gradient. Doesn't the same principle apply to RL problems? Is the randomness/noise introduced actually beneficial to the learning process? Am I missing something, or are these sources all wrong?
Not all the sources rely on stochastic gradient descent: e.g. keras-rl seems to rely on minibatches (https://github.com/keras-rl/keras-rl/blob/master/rl/agents/dqn.py)