0
$\begingroup$

Consider the following two excerpts from the research paper titled Densely Connected Convolutional Networks by Gao Huang et al.

#1: From abstract

Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output.

#2: From discussion

One explanation for the improved accuracy of dense convolutional networks may be that individual layers receive additional supervision from the loss function through the shorter connections. One can interpret DenseNets to perform a kind of “deep supervision”.

Both excerpts mention the type of connections called shorter connections, especially to the layers that are close to the input and the output layers of the deep convolutional neural network. What does it mean by shorter connections here?

$\endgroup$

2 Answers 2

1
$\begingroup$

I think this is one of those vague terminologies used in the context of skip (long-range connections). In a standard feedforward network, say if the information needs to propagate from layer 1 to 4, it has to go through two intermediate layers, namely $1 \to 2\to 3\to 4$. In a densely connected convolution network, every layer is connected to every layer downstream, meaning that you have a direct connection from layer 1 to 4, or $1\to 4$, hence a "shorter connection".

Note you also have other "shorter connections" such as $1\to 3\to 4$ and $1 \to 2 \to 4$

$\endgroup$
0
$\begingroup$

Shorter connections are nothing but concatenating output tensors along the channel axis. Having shorter connections between layers, will help in gradient flow throughout the network easily and hence better training.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .