I'm trying to study how backpropagation works step by step in a MultiLayer Perceptron neural network. I would really like to be able to understand how these calculations work. And I have a specific question I would like to ask. The formulas I'm trying to learn are the following to calculate the delta, and then to update the weights and bias:
Please, could you help me understand if is correct ?
In Output Layer
Calculate delta: error * derivative
Update weights: newW = oldW + (delta * neuron entry * learning_rate)
Update Bias: newB = oldB + (delta * learning_rate)
(Here in the output layer, the neuron's error is the difference between the desired output and the predicted output of the neuron, in the feedforward phase)
In the Hidden Layer:
Calculate delta: SUM of (delta * connection weight ) * derivative
Update weights: newW = oldW + (delta * neuron entry * learning_rate)
Update Bias: newB = oldB + (delta * learning_rate)
(In the SUM, Here, in the hidden layer, the error of a neuron is a summation, adding the delta multiplied by the connection weight of the neuron in this next layer with the neuron in the current layer, of each neuron in the next layer, that is, the connection weights that connect a neuron in the current hidden layer with a neuron in the next layer.)
I would like to ask you a specific question about these two formulas used in backpropagation, because I have this doubt: the formulas I am studying to calculate the delta and then to update the weights in the output layer and hidden layer are correct, Are these formulas I mentioned correct?
Please, could someone help me, and tell me if what I am studying is correct?
What are the backpropagation correct formulas?
