Capsule Networks use an encoder-decoder structure, where the encoder part consists of the capsule layers (PrimiaryCaps and DigitCaps) and is also the part of the capsule network which performs the actual classification. On the other hand, the decoder attempts to reconstruct the original image from the output of the correct DigitCap. The Decoder in the Capsule Network is used as the regularizer of the network as it helps the network learn better features.

I can see how the decoder is helpful for datasets such as MNIST where all image classes have clear differences and the input size if the image is quite small. However, if the input has large dimensions and differences between image classes are quite small, I see the decoder network as overkill, as it will find it hard to reconstruct images for different classes.

In my case, my dataset consists of 3D MRI images of patients which have Alzheimer's Disease and those who do not. I am down-sampling the images and producing 8 3D patches which will be used as input to the network. The patches still have high dimensions considering that these are 3D, and there are not many clear differences between patches of the two image classes.

My questions here are:

  1. How significant is the decoder part of the capsule network? CNNs that perform image classification, usually do not have a decoder part. Why does the capsule network rely on the decoder to learn better features?

  2. Are there any alternatives to the decoder within the capsule network, acting as a regularizer? Can the decoder be ignored completely?



You must log in to answer this question.

Browse other questions tagged .