New answers tagged sutton-barto
1
When the authors write go from
$$\nabla_{\theta}J \propto \sum_s \mu(s) \sum_a q_{\pi}(s,a)\nabla_{\theta}\pi(a|s;\theta)\;$$
to
$$\nabla_{\theta}J = E_{\pi}\left[\sum_a q_{\pi}(S_t,a) \nabla_{\theta}\pi(a|S_t;\theta)\right]\;$$
they are simply taking an expectation where the only random variable is the state $S_t$. This is because, as they say in the book, ...
Top 50 recent answers are included
