0
$\begingroup$

The DenseNet architecture can be summarize with this figure : enter image description here

Why there is transition layers between each blocks ?

In the papers, they justify the use of transition layers as follow :

The concatenation operation used in Eq. (2) is not viable when the size of feature-maps changes. However, an essential part of convolutional networks is pooling layers that change the size of feature-maps. To facilitate pooling in our architecture we divide the net- work into multiple densely connected dense blocks

But, if I understand what they means : the problem is that the feature map size can change, thus we can't concatenate. But how adding transition layer change this problem ?

And how can several dense blocks connected like this are more efficient that one single bigger dense block ?

Optional question : Why all standard DenseNet are made of 4 dense blocks ? I guess I will have the answer to this question if I understood better the previous questions...

$\endgroup$
1
$\begingroup$

The point of DenseNet was to go as deep as Resnet if not deeper and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context(which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map which I guess would be computationally very expensive and not feasible.

About transition layers(convolution + pooling) , I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from 56x56 to 28x28 to 14x14 and so on.

The authors state it this way,

To further improve model compactness, we can reduce the number of
feature-maps at transition layers

$\endgroup$
  • $\begingroup$ Thanks. So these transitions layers are here just to reduce complexity ? What is the price of this reduction ? Less precision ? If we had a really efficient computer, wouldn't it be better to have a single fully connected block ? So every feature map is accessible anywhere in the network ? $\endgroup$ – Astariul Oct 15 '18 at 8:37
  • 1
    $\begingroup$ Theoretically seems yes, we should do better with fully connected, but we might have to find that out with implementation which doesn't look feasible practically! This is just my conjecture. $\endgroup$ – caissalover Oct 15 '18 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.