2
$\begingroup$

I am following the OpenAI's spinning up tutorial Part 3: Intro to Policy Optimization. It is mentioned there that the reward-to-go reduces the variance of the policy gradient. While I understand the intuition behind it, I struggle to find a proof in the literature.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Does the answer to this question answer yours as well? $\endgroup$
    – user5093249
    Jun 10, 2020 at 13:55
  • $\begingroup$ No, the linked question only proofs that the reward-to-go does not introduce any bias to the gradient estimate. $\endgroup$
    – sirKris van Dela
    Jun 10, 2020 at 14:14
  • $\begingroup$ This is nontrivial to prove, actually anything involving stochastic function approximation is nontrivial. You can search research papers, you won't find it in any book right now $\endgroup$
    – FourierFlux
    Jun 10, 2020 at 14:33

0

Reset to default

You must log in to answer this question.