0
$\begingroup$

I am trying to implement the DDPG algorithm.

Why is the actor's loss calculated as the negative mean of the model's predicted Q values in the states we are in? Shouldn't it be like the difference of the Q values when random action is taken and Q values of the model's predicted actions in that state?

Basically, if a random action is generating a better Q value, only then the actor Q value should be updated, else we already have the value and no need to propagate actor loss separately. It will be taken care of in the value loss.

As I understand it, there may be a case when we want to maximize the actor Q values with a constraint given by backpropagation of value loss in the critic network. Is it correct?

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.