Skip to main content
added 21 characters in body
Source Link
nbro
  • 41.4k
  • 12
  • 114
  • 205

The point of DenseNet was to go as deep as ResnetResNets, if not deeper, and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context  (which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map, which, I guess, would be computationally very expensive and not feasible.

About transition layers  (convolution + pooling)  , I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from 56x56$56 \times 56$ to 28x28$28 \times 28$ to 14x14$14 \times 14$, and so on.

The authors state it this way,

To further improve model compactnessfurther improve model compactness, we can reduce the number of
featurecan reduce the number of feature-maps at transitionat transition layers

The point of DenseNet was to go as deep as Resnet if not deeper and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context(which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map which I guess would be computationally very expensive and not feasible.

About transition layers(convolution + pooling)  , I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from 56x56 to 28x28 to 14x14 and so on.

The authors state it this way,

To further improve model compactness, we can reduce the number of
feature-maps at transition layers

The point of DenseNet was to go as deep as ResNets, if not deeper, and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context  (which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map, which, I guess, would be computationally very expensive and not feasible.

About transition layers  (convolution + pooling), I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from $56 \times 56$ to $28 \times 28$ to $14 \times 14$, and so on.

The authors state it this way

To further improve model compactness, we can reduce the number of feature-maps at transition layers

Source Link

The point of DenseNet was to go as deep as Resnet if not deeper and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context(which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map which I guess would be computationally very expensive and not feasible.

About transition layers(convolution + pooling) , I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from 56x56 to 28x28 to 14x14 and so on.

The authors state it this way,

To further improve model compactness, we can reduce the number of
feature-maps at transition layers