1
$\begingroup$

In policy gradient theory subtracting state dependent baseline from Q(s,a) does not affect gradient of the objective function. I understand the proof shown below. One things is, if it's not affecting the gradient, then how this approach is better? Proof that state dependent baseline does not affect gradient of the objective funtion

$\endgroup$
2
  • $\begingroup$ Short answer: variance reduction. Some details may be found, e.g., here. $\endgroup$
    – mikkola
    Sep 20, 2022 at 14:11
  • $\begingroup$ @mikkola thank you for your time end effort sir. I will check this out. $\endgroup$ Sep 20, 2022 at 15:48

0

You must log in to answer this question.

Browse other questions tagged .