Questions tagged [contrastive-learning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer

Definition of negatives in NT-Xent loss

I'm trying to understand few details about NT-Xent loss defined in SimCLR paper(link). The loss is defined as $$\mathcal{l}_{i,j} = -\log\frac{\exp(sim(z_i,z_j)/\tau)}{\sum_{k=1}^{2N}\mathbb{1}_{[k\...
James Arten's user avatar
0 votes
0 answers

Why the positive example is included in the denominator of NT-Xent loss?

I have a little perplexity about the NT-Xent loss employed in self-supervised contrastive learning. What we are essentially doing is maximizing the similarity of pairs of augmented images while ...
James Arten's user avatar
0 votes
0 answers

Explicit representation learning task benefits than simply considering last nerual network layer

Neural networks are ineherently representation learners, so one could simply extract the last layer embedding $\textbf{z} \in \mathbb{R}^d$ of a neural network model and consider it as a ...
James Arten's user avatar
6 votes
1 answer

What is the difference between the triplet loss and the contrastive loss?

What is the difference between the triplet loss and the contrastive loss? They look same to me. I don't understand the nuances between the two. I have the following queries: When to use what? What ...
Exploring's user avatar
  • 253
1 vote
1 answer

Embedding Quality of Transfer Learning model vs Contrastive learning model

I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances. The downstream task is a classification problem. ...
Raj Rajeshwari Prasad's user avatar
2 votes
2 answers

Why does triplet loss allow to learn a ranking whereas contrastive loss only allows to learn similarity?

I am looking at this lecture, which states (link to exact time): What the triplet loss allows us in contrast to the contrastive loss is that we can learn a ranking. So it's not only about similarity, ...
Gulzar's user avatar
  • 729
1 vote
1 answer

What is the difference between Mean Teacher and Knowledge Distillation?

I recently read two papers: BYOL Bootstrap your own latent: A new approach to self-supervised Learning DINO Emerging Properties in Self-Supervised Vision Transformers. I am confused about the terms ...
Đặng Huy Hoàng's user avatar
1 vote
0 answers

How to use K-means clustering to visualise learnt features of a CNN model?

Recently, I was going through the paper Intriguing Properties of Contrastive Losses. In the paper (section 3.2), the authors try to determine how well the SimCLR framework has allowed the ResNet50 ...
VEDANT JOSHI's user avatar