As you might know, Capsule Networks have been recently introduced by Hinton. There also have been several heads up within his talks.

As expected, the paper elaborates on the idea way theoretically! However, as a fan of Occam's razor, I was wondering if anybody can simplify the idea behind the Capsule Networks or CapseNets.



One of the major advantages of Convolutional neural networks is their invariance to translation. However this invariance comes with a price and that is, it does not consider how different features are related to each other. For example, if we have a picture of a face CNN will have difficulties distinguishing relationship between mouth feature and nose features. Max pooling layers are the main reason for this effect. Because when we use max pooling layers, we lose the precise locations of the mouth and noise and we cannot say how they are related to each other.

Capsules try to keep the advantage of CNN and fix this drawback in two ways;

  1. Invariance: quoting from this paper

When the capsule is working properly, the probability of the visual entity being present is locally invariant – it does not change as the entity moves over the manifold of possible appearances within the limited domain covered by the capsule.

In other words, capsule takes into account the existence of the the specific feature that we are looking for like mouth or nose. This property makes sure that capsules are translation invariant the same that CNNs are.

  1. Equivariance: instead of making the feature translation invariance, capsule will make it translation-equivariant or viewpoint-equivariant. In other words, as the feature moves and changes its position in the image, feature vector representation will also change in the same way which makes it equivariant. This property of capsules tries to solve the drawback of max pooling layers that I mentioned at the beginning.
  • $\begingroup$ Nice answer! I see you've also answered this similar question - in the future, if you find a duplicate question, please cast the appropriate close vote/flag rather than posting the same answer to both questions. Thanks! $\endgroup$ – Ben N Nov 12 '17 at 16:27

Capsule Networks have two key ideas:

  • the first idea is how to represent multi-dimensional entities. Capsule Networks does this by grouping these properties of a feature together ("capsules").
  • the second is that you activate higher-level features by agreement between lower-level features ("routing by agreement").

First, Capsule Networks partition the image into regions subsets.

For each of these regions, it assumes that there is at most one instance of a single feature, called a Capsule.

A Capsule is able to represent an instance of a feature (but only one) and is able to represent all the different properties of that feature, e.g., its (x,y) coordinates, its colour, its movement etc.

The difference from Convolutional Neural Networks (CNNs) is that the Capsules bundle the neurons into groups with multi-dimensional properties, whereas in CNNs the neurons represent single, unrelated scalar properties.

This structured Capsule representation allows you to do "routing by agreement".

To understand this, lets look at the example of a face detector. Here, you could have capsules representing "mouth", "eye", "nose" etc. Since the Capsules are multi-dimensional you also train them to predict the parameters for the entire face.

Now, if the "mouth", "nose" and "eye" Capsules agree about the parameters of the face we have a very strong signal that this is a good prediction since accidental agreement in a high-dimensional space like a neural network is very unlikely.

You use this to stack the Capsules into deep networks where the activation of higher-level Capsules are conditioned on agreement between the lower-level Capsules (e.g. the Face capsule being activated by agreement on the face position between the Nose, Mouth, Eye Capsules in the earlier, lower-level layer).

In contrast to regular feed-forward nets this requires a bit of iteration in the forward pass through the network, but you can still use back-propagation train it.

It is an interesting way to add a bit of structure to the data. So far, it looks like they are able to provide better generalization from limited training data.


Not the answer you're looking for? Browse other questions tagged or ask your own question.