Questions tagged [contrastive-learning]
The contrastive-learning tag has no usage guidance.
8
questions
1
vote
1
answer
62
views
Definition of negatives in NT-Xent loss
I'm trying to understand few details about NT-Xent loss defined in SimCLR paper(link). The loss is defined as
$$\mathcal{l}_{i,j} = -\log\frac{\exp(sim(z_i,z_j)/\tau)}{\sum_{k=1}^{2N}\mathbb{1}_{[k\...
- 297
0
votes
0
answers
32
views
Why the positive example is included in the denominator of NT-Xent loss?
I have a little perplexity about the NT-Xent loss employed in self-supervised contrastive learning.
What we are essentially doing is maximizing the similarity of pairs of augmented images while ...
- 297
0
votes
0
answers
12
views
Explicit representation learning task benefits than simply considering last nerual network layer
Neural networks are ineherently representation learners, so one could simply extract the last layer embedding $\textbf{z} \in \mathbb{R}^d$ of a neural network model and consider it as a ...
- 297
6
votes
1
answer
5k
views
What is the difference between the triplet loss and the contrastive loss?
What is the difference between the triplet loss and the contrastive loss?
They look same to me. I don't understand the nuances between the two. I have the following queries:
When to use what?
What ...
- 253
1
vote
1
answer
100
views
Embedding Quality of Transfer Learning model vs Contrastive learning model
I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances.
The downstream task is a classification problem.
...
2
votes
2
answers
595
views
Why does triplet loss allow to learn a ranking whereas contrastive loss only allows to learn similarity?
I am looking at this lecture, which states (link to exact time):
What the triplet loss allows us in contrast to the contrastive loss is
that we can learn a ranking. So it's not only about similarity, ...
- 729
1
vote
1
answer
573
views
What is the difference between Mean Teacher and Knowledge Distillation?
I recently read two papers:
BYOL Bootstrap your own latent: A new approach to self-supervised Learning
DINO Emerging Properties in Self-Supervised Vision Transformers.
I am confused about the terms ...
1
vote
0
answers
139
views
How to use K-means clustering to visualise learnt features of a CNN model?
Recently, I was going through the paper Intriguing Properties of Contrastive Losses. In the paper (section 3.2), the authors try to determine how well the SimCLR framework has allowed the ResNet50 ...
- 31