3
$\begingroup$

Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)?

image

If so could you give some names or links to research papers?

Improve this question
$\endgroup$
2
  • $\begingroup$ This seems to be a duplicate of ai.stackexchange.com/q/17190/2444. Well, at least, I answer your question there too. $\endgroup$
    – nbro
    Commented Oct 29, 2020 at 9:17
  • $\begingroup$ @nbro Well, I don't think this question is a duplicate, but yes there is an answer to my question in the linked one. Thanks! $\endgroup$
    – GKozinski
    Commented Oct 29, 2020 at 9:41

1 Answer 1

Reset to default
2
$\begingroup$

This type of connections are called skip or residual connections. There are numerous works which employs this type of mechanism, for example: ResNet, SkipRNN. In addition here you can find a paper that empirically explores the skip connections for sequential tagging, or this one for speech enhancement.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .