How do CNNs handle inputs of different sizes and shapes?

Question

I am new to deep learning so feel free to correct me where I am wrong.

Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over this input. As you know, it is not possible to do this.

Also, CNNs have a fixed input shape(Correct me if I am wrong) or at least the input should be of the multiples of the CNN's intended input shape(e.g., 112 * 112, 224 * 224, etc)(Although the situation that this may work is rare.)

According to this PyTorch page, ResNet (for example) accepts images of any size as long as they are bigger than 224.

So my question is, how does it handle images of different sizes? Does it dynamically tweak parts of the structure (e.g., kernels, strides, paddings) based on the input? If yes, wouldn't that change the network architecture? Or it changes the input sizes to the intended size automatically?

Also, this does not answer my question.

Answer 1

There is a transformation applied to the image before fed through the neural network described in the 3rd codeblock from that page:

preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

This will first resize every image (regardless of size) and then crop the centre of the image, so the input to the NN is always the same size.

Also, you mentioned that an input of shape 7x7 cannot be convolved with a 3x3 filter with padding zero and stride 3, but that is possible. Let's say this is the original image (grayscale, so no channels):

$$ \begin{matrix} . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

The size 3x3 kernel with stride 3 and padding 0 moves through this as follows:

$$ \begin{matrix} K & K & K & . & . & . & . \\ K & K & K & . & . & . & . \\ K & K & K & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

Then

$$ \begin{matrix} . & . & . & K & K & K & . \\ . & . & . & K & K & K & . \\ . & . & . & K & K & K & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

Then

$$ \begin{matrix} . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ K & K & K & . & . & . & . \\ K & K & K & . & . & . & . \\ K & K & K & . & . & . & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

Finally,

$$ \begin{matrix} . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & K & K & K & . \\ . & . & . & K & K & K & . \\ . & . & . & K & K & K & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

Your output has a shape of 2x2.

Check out the formula for calculating the next layer sizes from https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html (scroll down to the Shape section) and the images on this repository to see how the kernel moves through the image: https://github.com/vdumoulin/conv_arithmetic.