# Total loss in backpropagation

I'd say I have some understanding of backpropagation, however I am not really sure of the total loss being calculated. Let us take the example below:

After 1 forward pass when I have to update the parameters, I basically have to find answers to the question - What is the change in loss with respect to the change in a certain parameter i.e. if I were to update parameter w5 then, I need to calculate $$\frac{\partial L}{\partial w_5} = \frac{\partial L}{\partial o_1} \cdot \frac{\partial o_1}{\partial w_5}$$ So how does the total loss that I calculated after a forward pass even matter? Is it simply for logging(keeping track of)?

Another doubt that I have is: What exactly is this graph indicating? If it is the Loss value for different set of parameter values, then shouldn't it be a discrete function?

• "So how does the total loss that I calculated after a forward pass even matter?" what do you mean by this? How would you go about computing the partial derivative values without the loss?
– Karl
Commented Mar 21 at 2:26

Decreasing total loss over gradient update steps indicates that the model is learning. For calculating the individual gradients, what matters though are the current values in the nodes. (Since $$\frac{\partial L}{\partial L} = 1.$$)