1
$\begingroup$

When doing semantic segmentation, we often make use of FCN, which can be thought of in two parts: an encoder and decoder. As I understand, the encoder compresses the image into a spatially small, but high number of channels. The decoder then uses this high channel activation map and upsamples it into a representation detailing the class for each pixel in the image.

My question is, why do we do this spatial compression at all?

For example, why would the architecture shown below be a bad choice:

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

There is never a 100% accurate theory, however it's been observed to be beneficial, however I would argue that is due to the following:

  • you want to have a latent dimension, to learn the manifold compressed version, so that the decoder has a harder life to overfit (kinda speculative)
  • when doing convolution, you can keep the height and width dimensions constant, by introducing artifacts (which obviously you don't really want), so you are forced to decrease the dimensionality in H and W... so as a trade off, you just compensate with the number of filters/channels

in other words, there is no correct answer, just empirical evidence that this "latent compressed version" works better in some cases

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .