1
$\begingroup$

Say we have the layer $X W + b = Y$.

  1. I want to get $\frac{dL}{dW}$ and we assume I have $\frac{dL}{dY}$. So all I need is to find $\frac{dY}{dW}$. I know that it should be $X^T\frac{dL}{dY}$ but don't understand why. please explain.
  2. I want to get $\frac{dL}{db}$ and we assume I have $\frac{dL}{dY}$. So all I need is to find $\frac{dY}{db}$. I know that it should be $\sum(\frac{dL}{dY})_i$ (I mean sum the rows) but I don't understand why. please explain.

Thanks :)

$\endgroup$
  • $\begingroup$ The TL;DR answer is that the partial derivative of $L$ with respect to the weights or bias is just an application of the chain rule. Maybe later I'll add a complete answer. $\endgroup$ – nbro Oct 22 '19 at 0:55
  • $\begingroup$ Yea I know that it is because of the chain rule. I’m looking for a concise clear proof. $\endgroup$ – Yuval Kirstain Oct 23 '19 at 5:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.