The paper, Deep Recurrent Q-Learning for Partially Observable MDPs, talks about stacking multiple observations in the input of a convolutional neural network.

How does this exactly work? Do the convolutional filters loop over each observation (image)?

(I know this isn't the right group to request this, but I'll highly appreciate if someone could also suggest a framework that helps with this.)

  • 1
    $\begingroup$ I didn't read that paper, but it seems like you're talking about creating inputs (which, I suppose, represent states of the environment) to the value network which are composed of $N > 1$ successive frames/images (i.e. a state is a sequence of successive frames). This was also done in the original DQN paper. This is done mainly to provide more context, i.e. to partially solve the "partial observability" problem. As far as I remember from DQN, you just stack the frames, and that's going to be treated as a single state to the value network, though that may not be actually a "state". $\endgroup$ – nbro Oct 28 '20 at 11:17
  • 1
    $\begingroup$ Anyway, in that specific paper, they may be doing something different (especially after having looked at figure 2). Maybe later, if, meanwhile, someone doesn't answer this question, I will quickly read it and provide an answer. $\endgroup$ – nbro Oct 28 '20 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.