I'm currently in the process of learning about using CNNs in image recognition. Many of the different resources I read that were explaining the motivation referred to the fact that these networks are (to some degree) translationally invariant. My understanding is as follows:
- Fully connected networks are ill-suited to image recognition because of the high dimensionality of the data and especially because they do not preserve the spatial relationships between pixels. I care about the state of the pixels surrounding pixel X as well as X itself.
- CNNs remedy this because they look at the image in 2 dimensions so that surrounding pixels are being considered as well (the kernel does this). However, while I care about what is around X, I don't care where X is. So, I apply the kernel the same everywhere and I make a bunch of layers so that I can get a bunch of kernels.
- I also use pooling. This reduces the data dimensions but also adds some of that translation invariance.
So, my question is: What feature of the CNN is causing the invariance? I saw some explanations saying it was a result of the maps basically activating when a certain feature shows up, regardless of where. Other said that the pooling meant that if a horizontal line, for instance, showed up in one place vs. a pixel over, the pooling would activate the same for both. It seems like the first reason would be totally sufficient and the second not really adequate but I could also see it being both.
I also read a paper (https://arxiv.org/pdf/1606.09549.pdf) about fully-convolutional networks. In section 2.1, the authors explain that they intentionally chose to make it fully-convolutional because that allowed them to "compute the similarity at all translated sub-windows on a dense grid in a single evaluation."
That sentence makes me think that it is more the first explanation. Anyways, I hope to gain a better intuitive understanding of how the different parts of the CNN come together to work particularly well on images. Thanks!