86 votes
Accepted

What is self-supervised learning in machine learning?

Introduction The term self-supervised learning (SSL) has been used (sometimes differently) in different contexts and fields, such as representation learning [1], neural networks, robotics [2], natural ...
user avatar
  • 33k
17 votes

What is self-supervised learning in machine learning?

Self-supervised learning is when you use some parts of the samples as labels for a task that requires a good degree of comprehension to be solved. I'll emphasize these two key points, before giving an ...
user avatar
  • 481
5 votes

How to generate labels for self-supervised training?

How can I generate the target label from the other data in the dataset? If you are asking how you can create the learning signal in SSL, when given an unlabelled dataset, for learning representations ...
user avatar
  • 33k
5 votes

What is self-supervised learning in machine learning?

Self-supervised visual recognition is often applied to representation learning. Here we first learn features on unlabeled data (representation learning), and then learn the real model on features ...
user avatar
  • 469
4 votes

What is feature embedding in the context of convolutional neural networks?

The term feature embedding appears to be a synonym for feature extraction, feature learning etc. I.e. a form of embedding/dimension reduction (with the caveat the goal may not be a lower dimensional ...
user avatar
4 votes
Accepted

What are examples of approaches to dimensionality reduction of feature vectors?

Dimensionality reduction could be achieved by using an Autoencoder Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the ...
user avatar
  • 360
4 votes
Accepted

Where do the feature extraction and representation learning differ?

Feature extraction (FE) is not the same as representation learning (RL), but they are similar and related. You describe accurately what feature extraction typically refers to, i.e. the process of ...
user avatar
  • 33k
3 votes

Can a neural network learn to predict a number given a binarized image of a rectangle?

It can definitely be learned, the question is the approach. It would be expensive and difficult from a modeling directive to do this Densely, so usually convolutions are the way to go. An issue with ...
user avatar
  • 2,229
3 votes

How to understand the concept of self-supervised learning in AI?

I don't think your interpretation is correct. Take images as example. Supervised Learning e.g. classification (maybe use CNN with a L2 loss function) Assume you have many images with different ...
user avatar
  • 226
2 votes

How to understand the concept of self-supervised learning in AI?

Andrew Zisserman, who is a pioneer in the field of self-supervised learning, described self-supervised learning in a talk at ICML as: Self-supervised Learning is a form of unsupervised learning ...
user avatar
2 votes
Accepted

When should I use feature learning as opposed to feature engineering?

manual feature engineering started becoming obsolete That is wrong. Any suggestion on when to use manual feature engineering, feature learning or a combination of the two? Deep learning is ...
user avatar
  • 1,007
2 votes
Accepted

How do we know that the neurons of an artificial neural network start by learning small features?

We do it experimentally; you're able to look at what each layer is learning by tweaking various values throughout the network and doing gradient ascent. For more detail, watch this lecture: https://...
user avatar
  • 1,324
2 votes

What are examples of approaches to dimensionality reduction of feature vectors?

Some examples of dimensionality reduction techniques: Linear methods Non-linear methods Graph-based methods("Network embedding") PCA CCA ICA SVD LDA NMF Kernel PCA GDA Autoencoders t-SNE ...
user avatar
2 votes
Accepted

Why different images of the same person, under some restrictions, are in a 50 dimension manifold?

The number 50 is essentially just a guess based on results when compressing and/or generating data of a certain type. The variables such as "the three translations of the body, the three ...
user avatar
  • 23.1k
1 vote

How do we know that the neurons of an artificial neural network start by learning small features?

The network architecture is relevant to this question. Convolutional neural network architectures enforce the building up of features because the neurons in earlier layers have access to a small ...
user avatar
1 vote

What is feature embedding in the context of convolutional neural networks?

Feature embeddings are basically anything that can act as a hidden representation for given object. In the case of images, a CNN architecture is built to create such hidden representation. Usually, ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible