So the code is related to using a buffer
class BufferWrapper(gym.ObservationWrapper): def __init__(self, env, n_steps, dtype=np.int): super(BufferWrapper, self).__init__(env) self.dtype = dtype old_space = env.observation_space self.observation_space = gym.spaces.Box(old_space.low.repeat(n_steps, axis=0), old_space.high.repeat(n_steps, axis=0), dtype=dtype) def reset(self): self.buffer = np.zeros_like(self.observation_space.low, dtype=self.dtype) return self.observation(self.env.reset()) def observation(self, observation): self.buffer[:-1] = self.buffer[1:] self.buffer[-1] = observation return self.buffer
It is used to basically do some image processing so that the DQN is fed some transformation of the image. This link provides some higher-level logic behind some operations.
How can I actually understand what's the reason behind the code? Almost all repos have the exact same lines with no explanation (e.g. atari games GitHub repo).
My specific question is what is the purpose of the line
self.buffer[-1] = observation?
In my case, my observation is a (7*1) array and I have to return that in an appropriate manner from the
The book has some mention of this class but I couldn't understand much from it