Using pre-trained models on image dataset that is totally different for object detection?

Question

I have been trying out various tutorials on object detection machine learning. All the tutorials so far have been to use a pre-trained model for practical reasons when detecting objects that the pre-trained model learnt (e.g cats & dogs). However, will this pre-trained model work if I input a few hundred images of a particular car engine part and predict this class, which the pre-trained model did not train on? Is it recommended to make a model from scratch in this case?

I am further confused by this in TensorFlow documentation (Images -> Transfer learning and fine-tuning), the summary states:

Using a pre-trained model for feature extraction: When working with a small dataset, it is a common practice to take advantage of features learned by a model trained on a larger dataset in the same domain

By that meaning, if I need to predict a particular car engine part then this statement seems to suggest I create a model from scratch?

TLDR: Will pre-trained model be able to work on image dataset that it has never learn before or better to work on a model from scratch?

Answer 1

Well, I think you forget about "fine tuning" stage here. What they mean in these tutorials is that you take such model that was pretrained on large dataset and you usually freeze from training all layers except the last one or few last ones and you train these last layer/layers on your smaller specific dataset. This is called "transfer learning".

So in theory model should learn more general features during that first trainin and use them without changes in the second training (fine tuning), when it learns features more specific to your problem. Only then you can use it for detection objects from your dataset. If the model has never seen a labeled part from your dataset during its training it will never come up what this is by itself that's why you need this fine tuning process. But fortunately it requires far less data and thanks to that is much faster too.

Usually you let fine tune only last layer. But if you feel that your problem is noticeably different from the images the model been trained on then you can try tuning two or three last layers.

It's not alaways easy to find a model pretrained on dataset from every possible domain so usually you just start with a model pretrained on datasets containing images of very general classes of objects such as ImageNet or COCO.

If you work on a model from scratch it would probably require a lot of data. It's easy to overfit deep learning models with just few hundreds samples.

Answer 2

Besides the answer from @GKozinski, you can also fine tune all the layers of the pretrained model.

Yes, it is incredibly commonplace to do transfer learning using pretrained models from different domains.