0
$\begingroup$

Lets say I want to fine-tune a model. I have a pretrained ResNet model and on top of this model I add some extra layers. And lets say I have a dataset of 10,000 images. The recommended way would be:

  1. Freeze the backbone and train the model
  2. Unfreeze the backbone and train the model with a low learning rate

My question now is which of these configurations is better.

Config 1

  1. Freeze the backbone and train the model with all 10,000 images of the dataset
  2. Unfreeze the backbone and train the model with a low learning rate with all 10,000 images.

Config 2

  1. Freeze the backbone and train the model, e.g. 2000 images of the dataset, so that we get rid of the initial randomness.
  2. Unfreeze the backbone and train the model with a low learning rate with all 10,000 images.

Config 3

  1. Freeze the backbone and train the model, e.g. 2000 images of the dataset, so that we get rid of the initial randomness.
  2. Unfreeze the backbone and train the model with a low learning rate with the rest of images (8000).

My intuition says that config 1 could be good for when the backbone was pretrained for cats, dog, bus, car, ship, etc. (like ImageNet or COCO) and my dataset is similar. But I think when I have a complete different dataset like (bio-)medical images or other kind of special and very different images, then config 2 or 3 could be a better choice since you get the weight in the right direction in the first step and then have more space for information to store in the second step.

I am curious to know if there is a preferred way of doing and whether there are good reasons for or against.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.