2
$\begingroup$

I have read a lot on Actor Critic and I'm not convinced that there is a qualitative difference doing direct gradient updates on the network and slightly adjusting a soft-max output in the direction of the advantage function and doing gradient descent on the error.

Can anyone explain why updating the gradient directly is necessary?

$\endgroup$
1
$\begingroup$

It might seem to give the same update direction but would it converge to desirable policy parameters?

Actor-Critic is proposed alongside the policy gradient theorem in Sutton 1999. It is shown to maximize the state-value function. If you are able to show that the technique of yours is, in fact, maximizing some desirable objective function, you could propose it with some soundness as well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.