# Why in RL function approximators with recurrent structures can learn planning?

In the paper An Investigation of Model-Free Planning the authors use ConvLSTM to learn a planning function. In particular, for each input x_t at time-step t the function approximator is run for 3 internal ticks. I wonder how these fake ticks help the learning. Also, when tested, the trained RL agent (with fozen weights) is given up to 10 no-op actions and this improves its predictions. Do you have an idea why that works? (seems like magic to me)

• You can use MathJax/latex on this site. I suggest you edit your post to use it! – nbro Jun 11 at 13:22