1
$\begingroup$

It seems to me that average pooling can be replaced by a strided convolution with a constant kernel. For instance, a 3x3 pooling would be equivalent to a strided convolution (of stride $3$) with a $3 \times 3$ matrix of constants, with each entry being $\frac{1}{9}$.

However, I haven't found any mention of this fact online (perhaps it's too trivial of an observation)? Why then are explicit pooling layers needed if they can be realized by convolutions?

$\endgroup$
1
$\begingroup$

Is average pooling equivalent to a strided convolution with a specific constant kernel?

Yes.

Why then are explicit pooling layers needed if they can be realized by convolutions?

It is probably because the convolution is more expensive than the usual/natural implementation (i.e. just summing and then dividing).

To see why, let's consider your example. If you implement pooling with a convolution (or cross-correlation), we would need to perform $9$ multiplications, then $8$ summations, for a total of $17$ operations. If we implement pooling as usual, we would need to perform $8$ summations and $1$ division (or multiplication), for a total of $9$ operations.

Moreover, convolution may also be more prone to numerical instability (multiplications of numbers in the range $[0, 1]$ are not nice), but I am not completely sure about this, given that we multiply always by the same numbers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.