Is my understanding of back-propogation correct?

Question

I am trying to learn backpropagation and this is what I know so far.

To update the weights of the neural network you have to figure out the partial derivative of each of the parameters on the loss function using the chain rule. List all of these partial derivatives in a column vector and you have your gradient vector of your current parameter's on the loss function. Then by taking the negative of the gradient vector to descend the loss function and multiplying it by the learning rate (step size) and adding it to your original gradient vector, you have your new weights.

Is my understanding correct? Also, how can this be done in iterations over training examples?

Answer 1

Your understanding seems to be correct (although your explanation isn't completely precise), apart from "adding it to your original gradient vector". You add the gradient vector to the parameters/weights vector.

(Note that back-propagation is just the algorithm that computes the gradient vector. The update of the parameters with the gradient of the loss function is the gradient descent/ascent step, even though it's true that some people, at least in the context of deep learning, refer to the combination of gradient descent and the computation of the gradients as the back-propagation algorithm, but this is just a terminology issue, which you should not dwell too much on.)

Also, how can this be done in iterations over training examples?

If I understand this question correctly, you want to know how we would update the parameters when there's more than one training example. In that case, you compute a gradient vector for each training example. Then the average of the gradient vectors (which is also a vector) is the actual gradient vector that you use to update the parameters. See this answer for more info.