I am trying to learn backpropagation and this is what I know so far.
To update the weights of the neural network you have to figure out the partial derivative of each of the parameters on the loss function using the chain rule. List all of these partial derivatives in a column vector and you have your gradient vector of your current parameter's on the loss function. Then by taking the negative of the gradient vector to descend the loss function and multiplying it by the learning rate (step size) and adding it to your original gradient vector, you have your new weights.
Is my understanding correct? Also, how can this be done in iterations over training examples?