People who code: we want your input. Take the Survey

Tag Info

Bootstrapped estimate is biased because it based on $V(s_{t+1})$ which is usually a biased estimate by some estimator such as neural network. I don't think this statement is totally correct in the context of the paper. Quoting the paper: Taking $\gamma < 1$ introduces bias into the policy gradient estimate, regardless of the value function’s accuracy. ...