4
$\begingroup$

I am new to transfer learning and I start by reading A Survey on Transfer Learning, and it stated the following:

according to different situations of labeled and unlabeled data in the source domain, we can further categorize the inductive transfer learning setting into two cases:

case $(a)$ (It is irrelevant to my question).

case $(b): $ No labeled data in the source domain are available. In this case, the inductive transfer learning setting is similar to the self-taught learning setting, which is first proposed by Raina et al. [22]. In the self-taught learning setting, the label spaces between the source and target domains may be different, which implies the side information of the source domain cannot be used directly. Thus, it’s similar to the inductive transfer learning setting where the labeled data in the source domain are unavailable.

From that, I understand that self-taught learning is inductive transfer learning.

But I opened the paper of self-taught learning that was mentioned (i.e paper by Raina et al. [22].), and It stated the following in the introduction:

Because self-taught learning places significantly fewer restrictions on the type of unlabeled data, in many practical applications (such as image, audio or text classification) it is much easier to apply than typical semi-supervised learning or transfer learning methods.

And here it looks like transfer learning is different from self-taught learning.

So what is the right relation between them?

$\endgroup$
2
  • 1
    $\begingroup$ I'm not familiar with the term self-taught learning, but I suspect it is related to self-supervised learning, which is turn is related to transfer learning, as SSL can be used for transfer learning. See this post for more info about self-supervised learning. $\endgroup$ – nbro May 11 at 0:09
  • $\begingroup$ @nbro It is good to know about self-supervised learning, Thank you. But what about the distribution of the data that used for it? and the actual labels of them? In Self_Taught learning, there is no need for the unlabeled data (that used for training) to follow the same class labels or generative distribution as the labelled data that will be used for the supervised task. $\endgroup$ – Kais Hasan May 11 at 8:47
1
$\begingroup$

After a lot of searches, I think self-taught learning is a Transfer learning category, I think when Self-taught learning paper published (2007), there isn't any good survey on transfer learning, and as seen by high citation of the paper of Pan, his paper (which published in 2009) describes transfer learning in a clear way that did not exist before it.

Also, it is reasonable to consider Self-Taught learning a transfer learning category because he actually transfers the knowledge learned from unlabelled data to use it in the supervised task that we want to do (there is no need for the unlabeled data (that used for training) to follow the same class labels or generative distribution as the labelled data that will be used for the supervised task).

If someone could find something wrong with my answer or there is something missing, then please tell me.

$\endgroup$
1
  • 2
    $\begingroup$ I added terminology tag to the main post, as I though it might be relevant. $\endgroup$ – DukeZhou May 10 at 23:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.