I'm trying to understand the chain rule of backpropagation. This is what I understood. Is it correct?

enter image description here

$$ \frac{\partial E }{ \partial w} = \sum_{i} \frac{\partial E }{ \partial a_i^{(l)} } (\sum_{j} \frac{\partial a_i^{(l)} }{ \partial a_j^{(l-1)} }(\sum_{k} \frac{\partial a_j^{(l-1)} }{ \partial a_k^{(l-2)}} \frac{\partial a_k^{(l-2)} }{ \partial z_k^{(l-2)}} \frac{\partial z_k^{(l-2)} }{ \partial w}))$$

  • $ a_i^{(l)}$ is the activation of the neuron $i$ in the $l$th layer
  • $ z_i^{(l)} $ is the sum of multiplication of weights and previous activations
  • $E$ is the error
  • $w$ is the weight
  • $\begingroup$ Could you please clarify: Is the variable $\omega$ a single weight connecting one input to one neuron in the hidden layer (layer $l-2$)? $\endgroup$ Jan 2 at 9:10
  • 1
    $\begingroup$ yes, it's the weight connecting the first input layer to the first neurone in the layer layer l−2 $\endgroup$ Jan 2 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.