To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category).
Few-shot learning seems to do the same, with maybe 5-20 images per category. Is that the only difference?
In both cases, we initially train the neural network with a large dataset, then fine-tune it with our custom datasets.
So, how is few-shot learning different from transfer learning?
They use the same techniques, but study different problems.
Transfer learning always does not imply that the novel classes have very-few samples (as few as 1 per class). Few-shot learning does.
The goal of transfer learning is to obtain transferrable features that can be used for a wide variety of downstream discriminative tasks. One example is using an ImageNet pretrained model as an initialization for any downstream task, but note that we need to train on large amounts of data on those novel classes for the model to be suitable to that task.
Note that you can't finetune an ImageNet classifier on few examples of COCO and expect it to generalize well, because it won't. It wasn't explicitly optimized for few-sample learning.
In few-shot learning, our aim is to obtain models that can generalize from few-samples. This could be transfer learned (with certain changes to the usual transfer learning scenario), or it could be meta-learned. It might not need both, it could just be augmented with data from the novel classes during the test time, and a classifier could be trained from scratch.
