Different methods of calculating gradients of cost function(loss function)

We require to find the gradient of loss function(cost function) w.r.t to the weights to use optimization methods such as SGD or gradient descent. So far, I have come across two ways to compute the gradient:

1. BackPropagation
2. Calculating gradient of loss function by calculus

I found many resources for understanding backpropagation. The 2nd method I am referring to is the image below(taken for a specific example, e is the error: difference between target and prediction): Also, the proof was mentioned in this paper:here

Moreover, I found this method while reading this blog.(You might have to scroll down to see the code: gradient = X.T.dot(error) / X.shape )

My question is are the two methods of finding gradient of cost function same? It appears different and if yes, which one is more efficient( though one can guess it is backpropagation)

Would be grateful for any help. Thanks for being patient(it's my 1st time learning ML).

• BackPropagation is called so because it uses the special chain rule of calculus to back propagate gradients and so yes both are same. Also, if other method exist to back propagate in a similar way it will also be called back propagation.
– user9947
Mar 31 '20 at 12:13
• But sir, if both are same, why do we use backPropagation, isn't it very lengthy Mar 31 '20 at 13:41
• Backpropagation is a method/process of propagating the loss back to previous layers, whereas gradient descent (and it's variants like SGD,RMSprop etc) is a method to optimize the neural network to reduce the loss. Loss is less when we "descent" to a valley point, which we calculate through gradients or slope (ie. derivatives) and we send this information back using backprop where different channels receive different value of the gradient depending on their contribution. We can also send info other than gradients using backprop, there are also methods which don't use one or other, or either May 4 '20 at 18:09