I am trying to understand how translation invariance is achieved in CNNs. For example, consider the following simple binary classification problem: predicting whether the letter that appears on an image is
We want our network to be translation invariant. That is, if we translate our input, the final output (i.e. the predicted probability) should be the same in both the original and the translated version of the image. Mathematically, we want:
$$y = f(T(x)) = f(x)$$
where $x$ is the original image and $T(x)$ its translated version. For example, in the following image:
our output $y$ must be the same.
Can someone explain how this translation invariance is achieved with the
My view (possibly wrong)
First, I will present how I understand the network achieves this translation invariance and then I will provide an explanation why we need pooling.
- Given that the feature maps of a CNN are equivariant (thanks to the convolution operation) as we translate the input these feature maps vary in the same way. That is, if a feature was present in position $x$ of the feature map, after we translate the image by $T$, it will appear now in position $T(x)$. Visualizing this:
And as we go deeper into the network where it learns to recognizes
B, these feature maps will follow the same behavior. So assuming multiple
Conv layers (reducing the dimensionality, e.g. from input
2x2) eventually we will get either a
black (presence of
A) or a
white pixel (not presence of
- The case where I think
MaxPoolingis useful is if we consider a slightly different input image such as in the following example.
\ edge has move slightly to the right compared to the
/. This slight translation is captured by the first
Conv1 layer. But what about the
Conv2 layer? Will it be able to detect the presence of the edge?
This might not be the case as shown in the final image. In the right figure we must detect a face, but if the relative positions are slightly changed (mouth is translated slightly downwards), then we might not detect the face at all.
However, if we use a
MaxPooling layer between
Conv2 then we will get the same feature map as in the case of the original image (putting the feature again into "focus" for the next layer).