I'm new to neural networks and trying to figure out its fundamentals but I cannot fully understand the back propagation algorithm.
In back propagation, I understand we want to go backwards from the last neurons to adjust the weights and biases to that predicted final neurons. For it to calculate error and derivative, It needs to have the last inputs, the predicted output based on the layer's weights and the actual value ( target ).
As In the final neuron layers we have all this information. But how do we calculate the inputs of middle and hidden layers?
Suppose we have the final output ( 0.73 ), we calculate the error and derivatives of W31, W32 and W33; and adjust them to match the final output, Then we shall go one layer back in our network.
Now we need the N11, N12, N13 and N14 values and the target values of N21, N22 and N23 to calculate errors and derivatives, but we don't have them
Should we feed forward the whole network and map all the labels and values of each neuron in memory to be able to access it later? Because it would be very, very memory and resource intensive on large networks.