2
$\begingroup$

I would like to incrementally train my model with my current dataset and I asked this question on Github issues, which is what I'm using SSD MobileNet v1: https://github.com/tensorflow/models/issues/7200#issuecomment-510850230

They guy told me about learning without forgetting. I'm now confused between learning without forgetting & transfer learning. How they differ from each other?

Paper on learning without forgetting: https://arxiv.org/abs/1606.09282

My initial problem, what I'm trying to achieve (mentioned in Github Issue) is the following.

I have trained my dataset on ssd_mobilenet_v1_coco model. I'm getting continuous incremental data. Right now, my dataset is very limited.

What I want to achieve is incremental training. So, as soon as I get new data, I can further train my already trained model and I don't have to retrain everything.

{ Save Trained Model -> New Data -> Train on Old model } Loop

Between its transfer learning essentially. Correct me if wrong. But how do I do it on my own saved model.

$\endgroup$
0
$\begingroup$

What I want to achieve is incremental training. So, as soon as I get new data, I can further train my already trained model and I don't have to retrain everything.

Learning without forgetting is one of the methods to solve multitask learning. If your model trained to solve problem A and then after sometimes you need your model to solve new problem B without forgetting the problem A (the model still good to solve the problem A), then you need this.

Transfer learning is a method to use a trained model to solve another task (and may forget the original task). For example, you use a model that originally trained to classify cat or dog to a new task that trying to classify goat or cow. You use this in hopes of speeding up your training process.

If your new data has the same task with the old data, you don't need to use multitask learning method. For example:

  • if your model trained with 50 images to detect an apple in the image, and then you get new 100 images to detect an apple then you just need to continue your train (incremental learning). In this case, you need (to save) the latest parameter of your model after trained (latest learning rate value, epoch, etc.), if you have it then you just need to run your training again (continue the epoch).
  • if your model trained with 100 images to detect an apple in the image, and then you get new 100 images to train your model to detect an orange and you don't care if your model will give a bad result to detect an apple, then you can use transfer learning. You may freeze a few first layers as "extractor" and initialize a new layer at the end.
  • if your model trained with 100 images to detect an apple in the image, and then you get new 100 images to detect an orange and your model must good to detect both apple and orange in an image, then you use multitask learning. The easiest method is to train your model with the apple+orange image, but you can also use another approach like proposed in Learning without Forgetting paper.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.