Questions tagged [reward-to-go]

For questions about the concept of "reward-to-go", which comes up e.g. in the context of policy gradients. The "expected reward-to-go for all states" is sometimes used as a synonym for "value function". See e.g. the paper "Learning the Variance of the Reward-To-Go" (2016) by Aviv Tamar et al. for more details.

Filter by
Sorted by
Tagged with
4
votes
1answer
1k views

Why does the “reward to go” trick in policy gradient methods work?

In the policy gradient method, there's a trick to reduce the variance of policy gradient. We use causality, and remove part of the sum over rewards so that only actions happened after the reward are ...
2
votes
0answers
125 views

What is the proof that “reward-to-go” reduces variance of policy gradient?

I am following the OpenAI's spinning up tutorial Part 3: Intro to Policy Optimization. It is mentioned there that the reward-to-go reduces the variance of the policy gradient. While I understand the ...