0
$\begingroup$

I'm trying to understand the DDPG algorithm shown at this page. I don't know what should the result of the gradient at step 14 be.

enter image description here

Is it a scalar that I have to use to update all the weights (so all weights are updated with the same value)? Or is it a list with a different values to use for updating for each weight? I'm used to working with loss functions and an $y$ target, but here I don't have them so I'm quite confused.

$\endgroup$
0
$\begingroup$

Each Q output is a scalar, so the sum of all those is a scalar. Thus, you're taking a gradient wrt your parameters of a scalar. The result is a vector with one entry per parameter.

| improve this answer | |
$\endgroup$
  • $\begingroup$ So to have those entries I have to compute the jacobian? $\endgroup$ – aandre_90 Jun 21 at 15:50
  • $\begingroup$ Yeah, basically. $\endgroup$ – harwiltz Jun 21 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.