Let's say I have a neural network with 5 layers, including the input and output layer. Each layer has 5 nodes. Assume the layers are fully connected, but the 3rd node in the 2nd layer is connected to the 5th node in the 4th layer. All these numbers are chosen at random for the example.
When is the 5th node in the 4th layer fed forward?
Let's go through it step by step. The first layer is normally fed forward to the second. The second layer is normally fed forward to the third, but the 3rd node is also fed forward to the 5th node of the 4th layer.
At this point, is the 5th node in the 4th layer now fed forward, or is it fed forward when the 3rd layer is done being fed forward? The 1st method would mean that the node would get fed forward 2 times and my concern is, if the output is still valid.
Furthermore, it would also come to 2 asynchronous outputs and how would these be interpreted?
Because in the brain, I heard, the neurons are fired when an impulse arrives so this would equal the 1st method.