Below is a quote from CS231n:
Prefer a stack of small filter CONV to one large receptive field CONV layer. Suppose that you stack three 3x3 CONV layers on top of each other (with non-linearities in between, of course). In this arrangement, each neuron on the first CONV layer has a 3x3 view of the input volume. A neuron on the second CONV layer has a 3x3 view of the first CONV layer, and hence by extension a 5x5 view of the input volume. Similarly, a neuron on the third CONV layer has a 3x3 view of the 2nd CONV layer, and hence a 7x7 view of the input volume. Suppose that instead of these three layers of 3x3 CONV, we only wanted to use a single CONV layer with 7x7 receptive fields. These neurons would have a receptive field size of the input volume that is identical in spatial extent (7x7), but with several disadvantages
My visualized interpretation:
How can you see through the first CNN layer from the second CNN layer and see a 5x5 sized receptive field?
There were no previous comments stating all the other hyperparameters, like input size, steps, padding, etc. which made this very confusing to visualize.
I think I found the answer. BUT I still don't understand it. In fact, I am more confused than ever.