New answers tagged

1

There is no sign error and we should not change to $\arg\max$. With Policy Gradients I find that it is not useful to think about things such as a 'loss'. In short, we want to first find the derivative of the RL objective $J(\theta) = v_\pi(s_0)$, where $\pi$ is our policy that depends on some parameters $\theta$. The policy gradient theorem tells us that $$\...


1

They are not maximizing the gradient, the gradient is of the form \begin{equation} \nabla_{\theta} J \approx \sum_{t=0}^T G_t \nabla_{\theta} \log(\pi_{\theta}(a_t|s_t)) \end{equation} that means that when implementing it in software you can form your objective as \begin{equation} J = \sum_{t=0}^T G_t \log(\pi_{\theta}(a_t|s_t)) \end{equation} and then ...


Top 50 recent answers are included