2
$\begingroup$

I was discussing the topic of self-supervised learning with a colleague. After a while we realized we were using different definitions. That's never helpful.

Both of us were introduced to self-supervised learning by reading or listening to Yann LeCun. He is renaming (part of) unsupervised learning to self-supervised learning. For example in this Facebook post.

Probably the definitions of unsupervised and self-supervised learning overlap. But to me the terms are not interchangeable. For example, a prototypical example of age-old unsupervised learning technique is k-means. To me that is unsupervised but not self-supervised learning.

Is Yann LeCun renaming the entire concept unsupervised learning to self-supervised learning? More specifically, is his opinion that we should call clustering and anomaly detection self-supervised learning? And in the limit, does he call k-means supervised-learning?

References are appreciated.

$\endgroup$
3
  • $\begingroup$ I don't know what LeCun thinks of SSL or how he uses this term, but, yes, SSL has been used to refer to different techniques, and we used to denote some of these as "unsupervised learning", so it's possible that someone calls k-means an SSL technique, given that, in a way, there's some kind of automatic learning signal, but, to be honest, it is really a stretch to call it SSL, as we don't really have or use any kind of label (e.g. this is a cluster) in k-means. It's not the same thing as training e.g. a denoising auto-encoder, where you really have the labels (i.e. original images). $\endgroup$ – nbro Apr 16 at 9:44
  • $\begingroup$ I just read that post that you're linking us to. I don't think he's calling k-means an SSL technique. He's using the term "SSL" in the same way that "SSL" is currently main used for and it's consistent with my description here. So, based just on that Facebook post, I don't think that LeCun implies that k-means is SSL. Not sure where you got this idea from, but it would be nice that you clarify this. $\endgroup$ – nbro Apr 16 at 9:50
  • 1
    $\begingroup$ Thanks for the feedback. I changed the question a bit to clarify why I introduced k-means. $\endgroup$ – Pieter Apr 16 at 10:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.