I am trying to implement the DDPG algorithm.

Why is the actor's loss calculated as the negative mean of the model's predicted Q values in the states we are in? Shouldn't it be like the difference of the Q values when random action is taken and Q values of the model's predicted actions in that state?

Basically, if a random action is generating a better Q value, only then the actor Q value should be updated, else we already have the value and no need to propagate actor loss separately. It will be taken care of in the value loss.

As I understand it, there may be a case when we want to maximize the actor Q values with a constraint given by backpropagation of value loss in the critic network. Is it correct?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.