Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer l, in a neural network to particular nodes in the previous layer l-1 of the neural network as well as to particular nodes in the next layer l+1 of the neural network?

This is obviously contrived, but here is a illustration of what I mean. enter image description here The thick line with arrow indecates an edge that leads to l-1 layer neuron

  • 3
    $\begingroup$ Yes. Feedback ANN and Recurrent ANN are types of what you want. $\endgroup$
    – OmG
    Jan 1 '18 at 8:11
  • 1
    $\begingroup$ Maybe make your question clearer with a diagram? I was not imagining RNNs as an answer, but ResNets with skip connections. Both RNNs and skip connections are useful, in very different ways, and could be explained in an answer. $\endgroup$ Jan 1 '18 at 13:15
  • 1
    $\begingroup$ Thanks for adding the diagram. This is closer to RNN in design as it involves loops, and thus necessarily some kind of time step parameter in order to decide which value to feed. I'm not sure how much that kind of feedback connection has been studied, and what name(s) it goes under though. $\endgroup$ Jan 2 '18 at 10:54
  • 1
    $\begingroup$ @OmG It looks like recurrent ANNs are the answer the question author was looking for, thanks! Would you like to expand your comment into an answer with some background/explanation? $\endgroup$
    – Ben N
    Jan 2 '18 at 18:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.