Questions tagged [variance-reduction]

For questions about variance reduction techniques in the context of artificial intelligence, in particular, sampling or Monte Carlo methods. An example of a variance reduction technique is Flipout, which was proposed in the paper "Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches" (2018) by Yeming Wen et al.

Filter by
Sorted by
Tagged with
0
votes
0answers
26 views

Which off-policy policy gradient estimator has lower variance?

Let $\pi_\theta$ be a target policy and $\beta_\theta$ be a behavior policy. I have seen the following 2 policy gradient estimators in the literature: $$ \operatorname*{E}_{\tau \sim \beta_\theta} \...
13
votes
3answers
7k views

Why does is make sense to normalize rewards per episode in reinforcement learning?

In Open AI's actor-critic and in Open AI's REINFORCE, the rewards are being normalized like so rewards = (rewards - rewards.mean()) / (rewards.std() + eps) on ...