Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)?


If so could you give some names or links to research papers?

  • $\begingroup$ This seems to be a duplicate of ai.stackexchange.com/q/17190/2444. Well, at least, I answer your question there too. $\endgroup$
    – nbro
    Oct 29, 2020 at 9:17
  • $\begingroup$ @nbro Well, I don't think this question is a duplicate, but yes there is an answer to my question in the linked one. Thanks! $\endgroup$
    – GKozinski
    Oct 29, 2020 at 9:41

1 Answer 1


This type of connections are called skip or residual connections. There are numerous works which employs this type of mechanism, for example: ResNet, SkipRNN. In addition here you can find a paper that empirically explores the skip connections for sequential tagging, or this one for speech enhancement.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .