3
$\begingroup$

Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)?

image

If so could you give some names or links to research papers?

$\endgroup$
2
  • $\begingroup$ This seems to be a duplicate of ai.stackexchange.com/q/17190/2444. Well, at least, I answer your question there too. $\endgroup$
    – nbro
    Oct 29 '20 at 9:17
  • $\begingroup$ @nbro Well, I don't think this question is a duplicate, but yes there is an answer to my question in the linked one. Thanks! $\endgroup$
    – Makintosz
    Oct 29 '20 at 9:41
2
$\begingroup$

This type of connections are called skip or residual connections. There are numerous works which employs this type of mechanism, for example: ResNet, SkipRNN. In addition here you can find a paper that empirically explores the skip connections for sequential tagging, or this one for speech enhancement.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.