Are artificial neurons in layer $l$ only affected by those in layer $l-1$ (providing inputs) or are they also affected by neurons in layer $l$ (and maybe by neurons in other layers)?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It depends on the architecture of the neural network. However, in general, no, neurons at layer $l$ are not only affected by neurons at layer $l-1$.
In the case of a multi-layer perceptron (or feed-forward neural network), only neurons at layer $l-1$ directly affect the neurons at layer $l$. However, neurons at layers $l-i$, for $i=2, \dots, l$, also indirectly affect the neurons at layer $l$.
In the case of recurrent neural networks, the output of neuron $j$ at level $l$ can also affect the same neuron but at a different time step.
In the case of residual networks, the output of a neuron at a layer $l-i$, for $i=2, \dots, l$, can directly affect the neurons at layer $l$. These non-neighboring connections are called skip connections because they skip layers.
There are probably other combinations of connections between neurons at different layers or the same layer.