3
$\begingroup$

I'm kinda new to machine learning and still not too solid on math and particularly calculus. I'm currently trying to implement PPO algorithm as described in the spiningUp website : enter image description here

This line is giving me a hard time :

enter image description here

What does the $\operatorname{argmax}$ mean, in this context? They are also talking about updating the policy with a gradient ascent. So, is taking argmax with respect to $\theta$ the same as doing:

enter image description here

where $J$ is the min() function?

$\endgroup$
0
$\begingroup$

In this case yes, $J$ is the big $\min$ expression and you apply Adam on that. But be careful because they say they do ascent, but automatic differentiation software usually minimizes given function so your $J$ would be $−\min(⋅)$.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.