Does anybody know what would happen if I changed input shape of pytorch models?

Question

In this https://pytorch.org/vision/stable/models.html tutorial it clearly states:

All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225].

Does that mean that for example if I want my model to have input size 128x128 it is or if I calculate mean and std which is unique to my dataset that it is gonna perform worse or won't work at all? I know that with tensorflow if you are loading pretrained models there is a specific argument input_shape which you can set according to your needs just like here:

tf.keras.applications.ResNet101(
include_top=True, weights='imagenet', input_tensor=None,
input_shape=None, pooling=None, classes=1000, **kwargs)

I know that I can pass any shape to those (pytorch) pretrained models and it works. What I wanna understand is can I change input shape of those models so that I don't decrease my models training performance?

Answer 1

Each machine learning model should be trained by constant input image shape, the bigger shape the more information that the model can extract but it also needs a heavier model.

A model's parameters will adapt with the datasets it learns on, which means it will perform well with the input shape that it learned. Therefore, to answer your question "What I wanna understand is can I change input shape of those models so that I don't decrease my models training performance?", the answer is no, it will decrease the performance.

"I know that I can pass any shape to those (pytorch) pretrained models and it works."

$\Rightarrow$ this happened because Pytorch team replace all Pooling layer with Adaptive Average Pooling 2d so you can pass any shape of the image into the model without any bugs.

Answer 2

In all pre-trained models, the input image has to be the same shape; the transform object resizes the image when you add it as a parameter to your dataset object

transform = T.Compose([T.Resize(256), T.CenterCrop(224), T.ToTensor()])
dataset = datasets.ImageNet(".", split="train", transform=transform)

T.Resize(256) changes the image shape then T.CenterCrop(224) takes a random crop to help with overfitting in the training data.

The mean and std are applied to each image before they are used as an input to the model. They were determined when the model was trained, I assume they were hyperparamters.