In the proof of the policy gradient theorem in the RL book of Sutton and Barto (that I shamelessly paste here):
there is the "unrolling" step that is supposed to be immediately clear
With just elementary calculus and re-arranging of terms
Well, it's not. :) Can someone explain this step in more detail?
How exactly is $Pr(s \rightarrow x, k, \pi)$ deduced by "unrolling"?