What is the fundamental difference between max pooling and adaptive max pooling used in PyTorch

Question

PyTorch provides max pooling and adaptive max pooling.

Both, max pooling and adaptive max pooling, is defined in three dimensions: 1d, 2d and 3d. For simplicity, I am discussing about 1d in this question.

For max pooling in one dimension, the documentation provides the formula to calculate the output.

In the simplest case, the output value of the layer with input size $(N,C,L)$ and output $(N,C,L_{out})$ can be precisely described as:

$$out(N_i,C_j,k) = \max\limits_{⁡m=0, \cdots ,kernel\_size−1} input(N_i,C_j,stride×k+m)$$

But, adaptive max pooling has no detailed explanation in the documentation.

What is the fundamental difference between max pooling and adaptive max-pooling? max-pooling expects kernel_size and stride as input but adaptive max-pooling does not expect them as inputs and asks only for output size, does it uses kernel and stride for performing the operation? If yes, how does it calculate both?

Answer 1

In PyTorch, max pooling operation and output size calculation differ between the two. For example, the maximum value is picked within a given window and stride to reduce tensor dimensions of the input in max pooling. Adaptive max pooling allows for more flexibility since it directly specifies the desired output size making the output match exactly to that size.Adaptive max pooling ensures a fixed output size unlike max pooling which needs manual specification of parameters.

Answer 2

Max Pooling and Adaptive pooling are used in CNN - Objective of Max pooling or adaptive pooling is to downsample the input feature map

Max pooling - Fixed - you provide kernel size and stride - Max pool produces output size based in input ,kernel ,stride

max_pool = nn.MaxPool2d(kernel_size=2, stride=2)

Adaptive Max pooling - Flexible you provide the expected output size - it takes care of window size and stride.

adaptive_pool = nn.AdaptiveMaxPool2d(output_size=(2, 2))

Preferences:

Max pooling focuses on local features, while adaptive pooling focuses on global information

Answer 3

Adaptive max pooling can be useful when the input dimensions are not fixed. Say, for images of different sizes.

Suppose you have a vector of length 10. Max pooling with kernel size 2 and adaptive pooling with output size 5 will do exactly the same thing because 5 is a multiple of 10. However, if you want the output size to be something other than a multiple of the input size you often can't use max pooling. For example, you can't max pool a 12-element vector into a 5-element vector. But you can use adaptive max pooling and it will interpolate the input into 5 slices and apply the reduction (max operation) onto those slices.

Since code says more than many words, here is Python code for one-dimensional adaptive max pooling:

def my_adaptive_max_pool1d(x, h):
    from math import floor, ceil
    sd = x.shape[1] / h
    O = []
    for i in range(h):
        lo = floor(sd * i)
        hi = ceil(sd * i + sd)
        O.append(max(x[0][lo:hi]))
    return torch.FloatTensor(O).unsqueeze(0)