Intuitively understanding translational invariance in CNNs

Question

I'm currently in the process of learning about using CNNs in image recognition. Many of the different resources I read that were explaining the motivation referred to the fact that these networks are (to some degree) translationally invariant. My understanding is as follows:

• Fully connected networks are ill-suited to image recognition because of the high dimensionality of the data and especially because they do not preserve the spatial relationships between pixels. I care about the state of the pixels surrounding pixel X as well as X itself.
• CNNs remedy this because they look at the image in 2 dimensions so that surrounding pixels are being considered as well (the kernel does this). However, while I care about what is around X, I don't care where X is. So, I apply the kernel the same everywhere and I make a bunch of layers so that I can get a bunch of kernels.
• I also use pooling. This reduces the data dimensions but also adds some of that translation invariance.

So, my question is: What feature of the CNN is causing the invariance? I saw some explanations saying it was a result of the maps basically activating when a certain feature shows up, regardless of where. Other said that the pooling meant that if a horizontal line, for instance, showed up in one place vs. a pixel over, the pooling would activate the same for both. It seems like the first reason would be totally sufficient and the second not really adequate but I could also see it being both.

I also read a paper (https://arxiv.org/pdf/1606.09549.pdf) about fully-convolutional networks. In section 2.1, the authors explain that they intentionally chose to make it fully-convolutional because that allowed them to "compute the similarity at all translated sub-windows on a dense grid in a single evaluation."

That sentence makes me think that it is more the first explanation. Anyways, I hope to gain a better intuitive understanding of how the different parts of the CNN come together to work particularly well on images. Thanks!

Answer 1

The translational invariance comes from the fact that the kernels are applied everywhere. Basically the kernels are "translated", i.e. shifted, and that means the feature they detect are detected everywhere.

Pooling may help with that in some cases, because in the end you need a dog at the right side to activate the same neuron as a dog on the left, so there needs to be funnelling of the information, but it is not strictly necessary. You could also imagine a CNN with a dog kernel in a late layer, which just recognises dogs everywhere, with no pooling necessary.

Fully connected networks are wasteful in that they don't share weights. To get the necessary translational invariance you therefore need to learn pretty much the same weights at all positions. But they do preserve the spatial relationship between pixels.

Pooling reduces the dimensionality, but it also forgets the precise position of a feature. This might help with generalisation, but I'm not sure whether that's a big factor.