0
$\begingroup$

I am new to deep learning so feel free to correct me where I am wrong.

Imagine this scenario where we have a 7 * 7 input. We want to slide a 3 * 3 filter with a stride of 3 and padding of zero over this input. As you know, it is not possible to do this.

Also, CNNs have a fixed input shape(Correct me if I am wrong) or at least the input should be of the multiples of the CNN's intended input shape(e.g., 112 * 112, 224 * 224, etc)(Although the situation that this may work is rare.)

According to this PyTorch page, ResNet (for example) accepts images of any size as long as they are bigger than 224.

So my question is, how does it handle images of different sizes? Does it dynamically tweak parts of the structure (e.g., kernels, strides, paddings) based on the input? If yes, wouldn't that change the network architecture? Or it changes the input sizes to the intended size automatically?

Also, this does not answer my question.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

There is a transformation applied to the image before fed through the neural network described in the 3rd codeblock from that page:

preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

This will first resize every image (regardless of size) and then crop the centre of the image, so the input to the NN is always the same size.

Also, you mentioned that an input of shape 7x7 cannot be convolved with a 3x3 filter with padding zero and stride 3, but that is possible. Let's say this is the original image (grayscale, so no channels):

$$ \begin{matrix} . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

The size 3x3 kernel with stride 3 and padding 0 moves through this as follows:

$$ \begin{matrix} K & K & K & . & . & . & . \\ K & K & K & . & . & . & . \\ K & K & K & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

Then

$$ \begin{matrix} . & . & . & K & K & K & . \\ . & . & . & K & K & K & . \\ . & . & . & K & K & K & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

Then

$$ \begin{matrix} . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ K & K & K & . & . & . & . \\ K & K & K & . & . & . & . \\ K & K & K & . & . & . & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

Finally,

$$ \begin{matrix} . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & . & . & . & . \\ . & . & . & K & K & K & . \\ . & . & . & K & K & K & . \\ . & . & . & K & K & K & . \\ . & . & . & . & . & . & . \\ \end{matrix} $$

Your output has a shape of 2x2.

Check out the formula for calculating the next layer sizes from https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html (scroll down to the Shape section) and the images on this repository to see how the kernel moves through the image: https://github.com/vdumoulin/conv_arithmetic.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks. Think I get it now. So in fact it can be convolved after all. But is this actually used? And is it possible to do transfer learning with an image with a different shape? I've seen guides that do such a thing but can't quite understand the idea. $\endgroup$
    – Ali Khalili
    Aug 29, 2021 at 7:12
  • $\begingroup$ It is possible. The transformations are not part of the NN, these are deterministic transformations on the source data. It is similar to the idea that if you have a photo frame of size 10cm x 10cm and you have a really high definition 8K photo, you can still print that 8K photo and put it in the photo frame, you just need to transform it to a smaller size before you do it. $\endgroup$
    – user42664
    Aug 29, 2021 at 7:26
  • $\begingroup$ Thanks, that sums it up. $\endgroup$
    – Ali Khalili
    Aug 29, 2021 at 7:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .