What are the differences between transfer learning and meta learning?

Question

What are the differences between meta-learning and transfer learning?

I have read 2 articles on Quora and TowardDataScience.

Meta learning is a part of machine learning theory in which some algorithms are applied on meta data about the case to improve a machine learning process. The meta data includes properties about the algorithm used, learning task itself etc. Using the meta data, one can make a better decision of chosen learning algorithm(s) to solve the problem more efficiently.

and

Transfer learning aims at improving the process of learning new tasks using the experience gained by solving predecessor problems which are somewhat similar. In practice, most of the time, machine learning models are designed to accomplish a single task. However, as humans, we make use of our past experience for not only repeating the same task in the future but learning completely new tasks, too. That is, if the new problem that we try to solve is similar to a few of our past experiences, it becomes easier for us. Thus, for the purpose of using the same learning approach in Machine Learning, transfer learning comprises methods to transfer past experience of one or more source tasks and makes use of it to boost learning in a related target task.

The comparisons still confuse me as both seem to share a lot of similarities in terms of reusability. Meta-learning is said to be "model agnostic", yet it uses metadata (hyperparameters or weights) from previously learned tasks. It goes the same with transfer learning, as it may reuse partially a trained network to solve related tasks. I understand that there is a lot more to discuss, but, broadly speaking, I do not see so much difference between the two.

People also use terms like "meta-transfer learning", which makes me think both types of learning have a strong connection with each other.

I also found a similar question, but the answers seem not to agree with each other. For example, some may say that multi-task learning is a sub-category of transfer learning, others may not think so.

Answer 1

First of all, I would like to say that it is possible that these terms are used inconsistently, given that at least transfer learning, AFAIK, is a relatively new expression, so, the general trick is to take terminology, notation and definitions with a grain of salt. However, in this case, although it may sound confusing to you, all of the current descriptions on this page (in your question and the other answers) don't seem inconsistent with my knowledge. In fact, I think I had already roughly read some of the cited research papers (e.g. the MAML paper).

Roughly speaking, although you can have formal definitions (e.g. the one in the MAML paper and also described in this answer), which may not be completely consistent across sources, meta-learning is about learning to learn or learning something that you usually don't directly learn (e.g. the hyperparameters), where learning is roughly a synonym for optimization. In fact, the meaning of the word "meta" in meta-learning is

denoting something of a higher or second-order kind

For example, in the context of training a neural network, you want to find a neural network that approximates a certain function (which is represented by the dataset). To do that, usually, you manually specify the optimizer, its parameters (e.g. the learning rate), the number of layers, etc. So, in this usual case, you will train a network (learn), but you will not know that the hyperparameters that you set are the most appropriate ones. So, in this case, training the neural network is the task of "learning". If you also want to learn the hyperparameters, then you will, in this sense, learn how to learn.

The concept of meta-learning is also common in reinforcement learning. For example, in the paper Metacontrol for Adaptive Imagination-Based Optimization, they even formalize the concept of a meta-Markov decision process. If you read the paper, which I did a long time ago, you will understand that they are talking about a higher-order MDP.

To conclude, in the context of machine learning, meta-learning usually refers to learning something that you usually don't learn in the standard problem or, as the definition of meta above suggests, to perform "higher-order" learning.

Transfer learning is often used as a synonym for fine-tuning, although that's not always the case. For example, in this TensorFlow tutorial, transfer learning is used to refer to the scenario where you freeze (i.e. make the parameters non-trainable) the convolution layers of a model $M$ pre-trained on a dataset $A$, replace the pre-trained dense layers of model $M$ on dataset $A$ with new dense layers for the new tasks/dataset $B$, then retrain the new model, by adjusting the parameters of this new dense layer, on the new dataset $B$. There are also papers that differentiate the two (although I don't remember which ones now). If you use transfer learning as a synonym for fine-tuning, then, roughly speaking, transfer learning is to use a pre-trained model and then slightly retrain it (e.g. with a smaller learning rate) on a new but related task (to the task the pre-trained model was originally trained for), but you don't necessarily freeze any layers. So, in this case, fine-tuning (or transfer learning) means to tune the pre-trained model to the new dataset (or task).

How is transfer learning (as fine-tuning) and meta-learning different?

Meta-learning is, in a way, about fine-tuning, but not exactly in the sense of transfer learning, but in the sense of hyperparameter optimization. Remember that I said that meta-learning can be about learning the parameters that you usually don't learn, i.e. the hyper-parameters? When you perform hyper-parameters optimization, people sometimes refer to it as fine-tuning. So, meta-learning is a way of performing hyperparameter optimization and thus fine-tuning, but not in the sense of transfer learning, which can be roughly thought of as retraining a pre-trained model but on a different task with a different dataset (with e.g. a smaller learning rate).

To conclude, take terminology, notation, and definitions with a grain of salt, even the ones in this answer.

Answer 2

Meta-learning is more about speeding up and optimizing hyperparameters for networks that are not trained at all, whereas transfer learning uses a net that has already been trained for some task and reusing part or all of that network to train on a new task which is relatively similar. So, although they can both be used from task to task to a certain degree, they are completely different from one another in practice and application, one tries to optimize configurations for a model and the other simply reuses an already optimized model, or part of it at least.

Answer 3

The difference really comes down to the fact that in meta-learning, there is a population of tasks $\tau$ which have distribution $p(\tau)$. The goal is to perform well on a task drawn from $p(\tau)$. Generally 'perform well' means that with only a few training steps or data points, the model can give good classification accuracy, achieve high reward in an RL setting, etc.

A concrete example is given in the original MAML paper 1, where the task is to perform regression on data given by a sinusoidal distribution with parameters $p(\theta)$. The meta-learning goal is to get high regression accuracy on tasks where the data is drawn from distributions coming from $p(\theta)$.

In contrast, transfer learning is a bit more general since there's not necessarily a notion of a distribution of tasks. There is generally just one (although there can be more) source problem $S$, and the goal is to do well on a target problem $T$. You know both of these explicitly, unlike in MAML where the goal is to do well amongst any unknown problem drawn from a certain distribution. Very often, this is performed by taking a model that performs well on $S$ and adapting it to work on $T$, perhaps by using extracted features from the model for $S$.

The extent to which this will succeed obviously depends on the similarity of the two tasks. This is also known in the literature as domain adaptation, and has some theoretical results 2, although the bounds are not really applicable to modern high-dimensional datasets.

1. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (Finn et al) 2017.

2. A Theory of Learning from Different Domains (Ben-David et al) 2010.

Answer 4

The way nbro describes meta-learning, it sounds identical to hyper-parameter optimization. Here I would like to clarify possible differences:

According to Dataset2Vec: learning dataset meta-features

Meta-learning, or learning to learn, refers to any learning approach that systematically makes use of prior learning experiences to accelerate training on unseen tasks or datasets. For example, after having chosen hyperparameters for dozens of different learning tasks, one would like to learn how to choose them for the next task at hand.

So, while hyper-parameter optimization often uses a single dataset, while meta-learning considers multiple datasets, where one (or multiple) are used to optimize hyper-parameters and then use those parameters for later datasets (without re-optimizing those hyper-parameters).

While this might feel a bit like "invent your own category" (a technique in marketing), meta-learning might be considerably more difficult. The reason is that (classical) hyper-parameter optimization only needs to build models that generalize over a single datasets, while meta-learning needs to generalize over multiple datasets - without seeing the later dataset(s) (at least for the hyper-parameter optimization).