1
$\begingroup$

I'm trying to understand the DDPG algorithm shown at this page. I don't know what should the result of the gradient at step 14 be.

enter image description here

Is it a scalar that I have to use to update all the weights (so all weights are updated with the same value)? Or is it a list with a different values to use for updating for each weight? I'm used to working with loss functions and an $y$ target, but here I don't have them so I'm quite confused.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Each Q output is a scalar, so the sum of all those is a scalar. Thus, you're taking a gradient wrt your parameters of a scalar. The result is a vector with one entry per parameter.

Improve this answer
$\endgroup$
2
  • $\begingroup$ So to have those entries I have to compute the jacobian? $\endgroup$
    – unter_983
    Commented Jun 21, 2020 at 15:50
  • $\begingroup$ Yeah, basically. $\endgroup$
    – harwiltz
    Commented Jun 21, 2020 at 18:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .