What would the receptive field of a neuron with a 2x2 max pool, stride of 3?

Question

Bit confused about the concept of a receptive field. I understand it to be the size of the region which influences the output of the neuron of interest. If I have an input of 16x16, and immediately pool afterwards (for simplicity) with a 2x2 window and a stride of 3, does the receptive field of an output neuron end up being 3x3? Or does it remain 2x2, since we do not draw upon any useful info with the gaps between the windows?

Answer 1

The receptive field of a neuron at a given depth (layer) is the number of input pixels that affect its output.

For example if you apply a $K\times K$ kernel at each layer, the receptive field of an element in the $L$-th activation would have a size of $$(1 + L(K-1))\times (1+L(K-1))$$ Now if you introduce a stride $S$ each time, the size of the receptive field is approximately divided by $S$ (you need to take into account padding, and rounding...)

In your specific example, you have a $2\times 2$ kernel (it doesn't matter whether the layer is a pooling or a conv) with $S=3$ as stride. This means that you discard $5$ pixels at each slide of the kernel: basically you have $3\times 3=9$ pixels each time but just consider $2\times 2=4$ of them, so five got lost.

So, at the input level, the receptive field of the pooling will still be $2\times 2$ but the large stride would have the effect of reducing the receptive field of the next layer(s).