2
$\begingroup$

To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category).

Few-shot learning seems to do the same, with maybe 5-20 images per category. Is that the only difference?

In both cases, we initially train the neural network with a large dataset, then fine-tune it with our custom datasets.

So, how is few-shot learning different from transfer learning?

$\endgroup$
2
  • 1
    $\begingroup$ Can you please put your main specific question in the title? "Few shot and transfer learning" is not a question and it's not specific. Take a look at this for the motivation. $\endgroup$
    – nbro
    Jan 7 at 10:38
  • $\begingroup$ I edited your post to remove the question "Can somebody help me understand how approaches like meta-learning are different from transfer learning?", because it's a completely different question than "How is few-shot learning different from transfer learning?". By the way, we already have that question here. I also try to clarify certain parts of your post. Please, make sure that I didn't change the meaning of the post. $\endgroup$
    – nbro
    Jan 7 at 17:23
1
$\begingroup$

They use the same techniques, but study different problems.

Transfer learning always does not imply that the novel classes have very-few samples (as few as 1 per class). Few-shot learning does.

The goal of transfer learning is to obtain transferrable features that can be used for a wide variety of downstream discriminative tasks. One example is using an ImageNet pretrained model as an initialization for any downstream task, but note that we need to train on large amounts of data on those novel classes for the model to be suitable to that task.

Note that you can't finetune an ImageNet classifier on few examples of COCO and expect it to generalize well, because it won't. It wasn't explicitly optimized for few-sample learning.

In few-shot learning, our aim is to obtain models that can generalize from few-samples. This could be transfer learned (with certain changes to the usual transfer learning scenario), or it could be meta-learned. It might not need both, it could just be augmented with data from the novel classes during the test time, and a classifier could be trained from scratch.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.