Questions tagged [representation-learning]

For questions related to feature learning (also known as representation learning), which is a set of techniques that can learn the features associated with the raw data. It is similar to feature engineering, but, in the case of feature learning, the features are learned and not handcrafted.

Filter by
Sorted by
Tagged with
1 vote
0 answers
23 views

Toy dataset: Radial VAE

I'm evaluating disentanglement in toy datasets seeing as we have such little understanding of the phenomena. I'm using various tools from differential geometry. Now I want to train a VAE on the ...
John Miller's user avatar
2 votes
0 answers
39 views

Does this learning scenario have a name? If so, can someone point me to relevant literature?

I am faced with a problem which I bet was already solved before, but that I had never seen. Perhaps by discussing it abstractly, someone can point me to relevant literature. It goes like this: I have ...
Alek Fröhlich's user avatar
0 votes
2 answers
83 views

BYOL: Why is there a prediction network in the online network but not in the target network?

In the BYOL paper, the following architecture is presented: Why is a prediction network added to the online network, which is not present in the target network? How are the online and target compared ...
Robin van Hoorn's user avatar
0 votes
0 answers
26 views

How are the softmax normalized weights in ELMo actually learned and computed?

I was reading the ELMo paper, and they speak of task-specific representations of words (or tokens generally speaking) by using the following equation: $ELMo_{k}^{task} = \gamma^{task}\sum_{0}^{L}{s_{j}...
Propr's user avatar
  • 1
0 votes
0 answers
25 views

How does the latent space learned by a neural network look like, in the regression case?

Neural networks are representation learners. For example, I understand that in the case of a classification task, the network will learn to cluster all points based on their classes, in a way that a ...
James Arten's user avatar
0 votes
0 answers
23 views

Is there a test to determine if a feature space is sufficient for a classification problem?

I am working with GNNs for a node classification problem, and I am only able to achieve about 50% accuracy for the training set. I am not able to overfit the model to the training data. This makes me ...
Ralff's user avatar
  • 173
0 votes
0 answers
12 views

Explicit representation learning task benefits than simply considering last nerual network layer

Neural networks are ineherently representation learners, so one could simply extract the last layer embedding $\textbf{z} \in \mathbb{R}^d$ of a neural network model and consider it as a ...
James Arten's user avatar
1 vote
1 answer
549 views

How to get meaningful vector embeddings for (lat, long) points and also GPS trajectories?

I have a data that consists of approx. 1.5M taxi trips in Porto, Portugal. (from: https://www.kaggle.com/competitions/pkdd-15-taxi-trip-time-prediction-ii/overview) Each of these trips have it's GPS ...
I am not a robot's user avatar
2 votes
1 answer
354 views

Order of features learned by DNNs during training?

I'm looking for papers probing into the question of what features get learned when (or equivalently what subproblems get "solved" when) during the training process. For example, a paper ...
jon_simon's user avatar
  • 121
2 votes
1 answer
117 views

Multi-objective training involving maximization of one loss function and minimization of another

I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a ...
ChargeShivers's user avatar
4 votes
2 answers
2k views

What is the difference between representation and embedding?

As I searched about this two terms, I found they are somehow like each other, both try to create a vector from raw data as I understood. But, what is the difference of this two term?
aliiiiiiiiiiiiiiiiiiiii's user avatar
1 vote
1 answer
348 views

How do neural networks learn specific features throughout the layers?

I was reading about convolutional neural networks and I came across such an explanation: Consider detecting features in human face. The earlier layers of neural networks learn coarse features such as ...
levitatmas's user avatar
4 votes
2 answers
55 views

Is there any proper literature on the types of features that different layers of a deep neural network learn?

Let's consider a deep convolutional network. It seems that there is some consensus on the following notions: 1. Shallow layers tend to recognise more low-level features such as edges and curves. 2. ...
mesllo's user avatar
  • 141
1 vote
0 answers
42 views

Is there a name for this model?

I have an image autoencoder model trained as follows: Step 1) train a GAN to obtain a generator capable of drawing from the data manifold by sampling a normal distribution in latent space Step 2) ...
user11305730's user avatar
1 vote
0 answers
24 views

In Graph Neural Network is Message Passing Step Agnostic of Output Values during Training?

So Graph Neural Networks is about representation learning where initially representation of graph is learned in the form of node embeddings. My question is: Are the output values back propagated and ...
user0193's user avatar
  • 145
0 votes
1 answer
85 views

A technique of aggregating many input images to a single representation of the relevant features within

I have a few thousand images and I would like to generate a representation of the foreground patterns within - a composition of all of its features, so to speak. In simple terms: take 10000 images of ...
mluerig's user avatar
  • 101
0 votes
1 answer
313 views

How can I adapt a trained neural network model to learn from newer data containing additional features?

We shall assume that we have a trained neural network model for some task $A$. The dataset used to train the model contained some $n$ features per sample. Using this dataset, we were able to train a ...
user52084's user avatar
1 vote
0 answers
14 views

Is there some way for us to know if the neural network internally finds an association between labels?

I have a question about the association between labels. Say my neural network performs multi-labeling in its output layer. Now, if one of the labels is for whether a person lives in city $X$, another ...
Abhiram Natarajan's user avatar
1 vote
0 answers
145 views

How to use K-means clustering to visualise learnt features of a CNN model?

Recently, I was going through the paper Intriguing Properties of Contrastive Losses. In the paper (section 3.2), the authors try to determine how well the SimCLR framework has allowed the ResNet50 ...
VEDANT JOSHI's user avatar
2 votes
1 answer
359 views

Does a bigger neural network learn "worse" representations than a small neural network when the amount of data isn't enough?

Assume we have a neural network and we want to train it on a classification problem. The hidden layers of the neural network are kind of feature representations of the input data. If the neural ...
realmarv's user avatar
0 votes
1 answer
75 views

Why disentangling the features of variation in representation?

Consider the following excerpt from abstract of the research paper titled Better Mixing via Deep Representations by Yoshua Bengio et al. It has been hypothesized, ...
hanugm's user avatar
  • 3,612
0 votes
0 answers
122 views

Train separate AutoEncoder's on each class or one AE for all classes to learn features?

I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I ...
Elise Le's user avatar
4 votes
1 answer
2k views

Where do the feature extraction and representation learning differ?

Feature selection is a process of selecting a subset of features that contribute the most. Feature extraction allows getting new features that are not actually present in the given set of features. ...
hanugm's user avatar
  • 3,612
3 votes
1 answer
70 views

Why different images of the same person, under some restrictions, are in a 50 dimension manifold?

In this lecture (starting from 1:31:00) the professor says that the set of all images of a person lives in a low dimensional surface (compared the the set of all possible images). And he says that the ...
Daviiid's user avatar
  • 573
3 votes
2 answers
111 views

How do we know that the neurons of an artificial neural network start by learning small features?

I'd like to ask you how do we know that neural networks start by learning small, basic features or "parts" of the data and then use them to build up more complex features as we go through ...
Daviiid's user avatar
  • 573
0 votes
2 answers
471 views

classification of unseen classes of image in open set classification

I have a scanned image, and they need to be classified in one of the pre-defined image classes, so that it can be sorted. However, the problem is the open nature of the classes. At testing time, new ...
Rambo_john's user avatar
2 votes
0 answers
63 views

Given a 2-layer GCN, can we choose the dimensions of the 2nd weight matrix, such that this architecture has the same capacity as a 1-layer GCN?

This might be more of a question about nested function classes: For $k$ class node classification in a graph with $n$ nodes, and $d$ feature vector. I want to compare Architecture I: the GCN model of ...
Tinatim's user avatar
  • 121
3 votes
1 answer
2k views

How to generate labels for self-supervised training?

I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image. Let's say that I have an image classification task, ...
Vesko Vujovic's user avatar
1 vote
0 answers
82 views

Extending patch based image classification into image classification

I am trying to classify tampered, pristine images from set of images, in that I have built a network in which I would divide the image into multiple overlapping patches and then classify them into ...
kiran's user avatar
  • 21
3 votes
1 answer
313 views

Does self-supervised learning require auxiliary tasks?

Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category "self-supervised": Some say, self-...
Make42's user avatar
  • 163
1 vote
0 answers
22 views

Convolutional Feature Encoding Methods in DCNN

In Computer Vision, feature encoding methods are used on pre-trained DCNN to increase the feature robustness to certain conditions such as viewpoint/appearance variations ref. I was just wondering if ...
doplano's user avatar
  • 299
2 votes
1 answer
151 views

Can a neural network learn to predict a number given a binarized image of a rectangle?

Let's assume that we have a regression problem. Our input is just binarized image that contains a single rectangle and we want to predict just a float number. Actually, this floating-point number ...
Bedrick Kiq's user avatar
5 votes
2 answers
180 views

What are examples of approaches to dimensionality reduction of feature vectors?

Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements. I would like to apply some augmentation techniques to reduce the ...
doplano's user avatar
  • 299
5 votes
2 answers
1k views

How to understand the concept of self-supervised learning in AI?

I am new to self-supervised learning and it all seems a little magical at the moment. The only way I can get an intuitive understanding is to assume that, for real-world problems, features are still ...
user3546025's user avatar
1 vote
0 answers
116 views

Plot class activation heatmap of Caffe Model in Python

Given the following 3 research papers, the authors have shown different heatmap graphical representations for features of the trained CNN models: On the performance of Convnet feature for place ...
doplano's user avatar
  • 299
2 votes
1 answer
226 views

What does "class-level discriminative feature representation" mean in the paper "Semi-Supervised Deep Learning with Memory"?

I am reading the paper Semi-Supervised Deep Learning with Memory (2018) by Yanbei Chen et al. The topic is the classification of images using semi-supervised learning. The authors use a term on page ...
IntegrateThis's user avatar
1 vote
0 answers
38 views

Is it possible to use adversarial training to learn invariant features?

Given a set of time series data that are generated from different sites where all sites are investigating the same objective but with slightly different protocols. Is it possible to use adversarial ...
Mwiza Kunda's user avatar
95 votes
3 answers
85k views

What is self-supervised learning in machine learning?

What is self-supervised learning in machine learning? How is it different from supervised learning?
nbro's user avatar
  • 39.5k
1 vote
1 answer
112 views

When should I use feature learning as opposed to feature engineering?

With the advancement of deep learning and a few others automated features learning techniques, manual feature engineering started becoming obsolete. Any suggestion on when to use manual feature ...
Ruchit Dalwadi's user avatar
5 votes
2 answers
8k views

What is feature embedding in the context of convolutional neural networks?

What are feature embeddings in the context of convolutional neural networks? Is it related to bottleneck features or feature vectors?
Kaustubh's user avatar