We require to find the gradient of loss function(cost function) w.r.t to the weights to use optimization methods such as SGD or gradient descent. So far, I have come across two ways to compute the gradient:
- Calculating gradient of loss function by calculus
I found many resources for understanding backpropagation. The 2nd method I am referring to is the image below(taken for a specific example, e is the error: difference between target and prediction):
Also, the proof was mentioned in this paper:here
Moreover, I found this method while reading this blog.(You might have to scroll down to see the code: gradient = X.T.dot(error) / X.shape )
My question is are the two methods of finding gradient of cost function same? It appears different and if yes, which one is more efficient( though one can guess it is backpropagation)
Would be grateful for any help. Thanks for being patient(it's my 1st time learning ML).