I am specifically talking about this proof Why does the "reward to go" trick in policy gradient methods work?

where Dennis says 'on ith iteration the outer sum of random variable and gradients are independent which means we are allowed to split them', my question is why can't we can do the same with final result of the proof where the only difference is we are summing rewards from t' = t onwards? of course we cannot do this as this would result in gradient J_theta to be zero, I just wanted to understand what am I missing here. Thanks



You must log in to answer this question.

Browse other questions tagged .