I quite often find projects using pre-trained model and using them as a starting point for their new model that learns something novel from thier dataset or on-live learning process - e.g. using a webcam or live audio.

Is this quite usual and recommended to speed up training a model? For example using a model trained on ImageNet as a first layer to your model that will categorise faces specifically.


Yes, it is recommended to start with pre-trained model if you don't have high-end hardware. You can use a pre-trained model for fine-tuning (their trained weight as your initial weight) or use it as feature extractor (you remove few last layers, and then train it).

Why we need a pre-trained network?

  • Because training a good deep model takes a lot of time and needs a lot of hardware. Some good models, like DenseNet, ResNet, even VGG16 need days of training. You can read from VGG original paper:

    On a system equipped with four NVIDIA Titan Black GPUs, training a single net took 2–3 weeks depending on the architecture.

  • Sometimes our case is similar to the dataset of pre-trained models out there. For example, if we need to classify some images of flower type, even our case has different class, we can use some first Convolutional layers of pre-trained model and use it as a feature extractor.

You can read more about transfer learning from this paper or this page.

  • $\begingroup$ Extremely useful thank you. So I guess part of the challenge is knowing what models are available and would suit your domain? $\endgroup$
    – benbyford
    Apr 19 '19 at 9:23
  • 1
    $\begingroup$ that's true, you can't just pick model randomly and then use it in your case. There is a simple rule to choose the right pre-trained model, if I remember correctly it's available on the page I mentioned $\endgroup$
    – malioboro
    Apr 19 '19 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.