3
$\begingroup$

I have read a lot on Actor Critic and I'm not convinced that there is a qualitative difference doing direct gradient updates on the network and slightly adjusting a soft-max output in the direction of the advantage function and doing gradient descent on the error.

Can anyone explain why updating the gradient directly is necessary?

$\endgroup$

1 Answer 1

1
$\begingroup$

It might seem to give the same update direction but would it converge to desirable policy parameters?

Actor-Critic is proposed alongside the policy gradient theorem in Sutton 1999. It is shown to maximize the state-value function. If you are able to show that the technique of yours is, in fact, maximizing some desirable objective function, you could propose it with some soundness as well.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .