1
$\begingroup$

The DenseNet architecture can be summarizde with this figure: enter image description here

Why there are transition layers between each block?

In the papers, they justify the use of transition layers as follow :

The concatenation operation used in Eq. (2) is not viable when the size of feature-maps changes. However, an essential part of convolutional networks is pooling layers that change the size of feature-maps. To facilitate pooling in our architecture we divide the network into multiple densely connected dense blocks

So, if I understood correctly, the problem is that the feature map size can change, thus we can't concatenate. But how does the addition of transition layers solve this problem?

And how can several dense blocks connected like this be more efficient that one single bigger dense block?

Furthermore, why are all standard DenseNets made of 4 dense blocks? I guess I will have the answer to this question if I understood better the previous questions.

$\endgroup$
1
$\begingroup$

The point of DenseNet was to go as deep as ResNets, if not deeper, and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context (which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map, which, I guess, would be computationally very expensive and not feasible.

About transition layers (convolution + pooling), I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from $56 \times 56$ to $28 \times 28$ to $14 \times 14$, and so on.

The authors state it this way

To further improve model compactness, we can reduce the number of feature-maps at transition layers

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks. So these transitions layers are here just to reduce complexity ? What is the price of this reduction ? Less precision ? If we had a really efficient computer, wouldn't it be better to have a single fully connected block ? So every feature map is accessible anywhere in the network ? $\endgroup$ – Astariul Oct 15 '18 at 8:37
  • 1
    $\begingroup$ Theoretically seems yes, we should do better with fully connected, but we might have to find that out with implementation which doesn't look feasible practically! This is just my conjecture. $\endgroup$ – caissalover Oct 15 '18 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.