1
$\begingroup$

I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances.

The downstream task is a classification problem.

Transfer Learning Due to limited data, I tried to use Transfer learning model trained on "Imagenet"(ResNet50 V2 "Deep Residual Learning for Image Recognition Kaiming He, et.al").

I used the embedding from the pretrained model and trained Linear SVM and achieved a F1 score of 0.84.

Contrastive Learning I also trained a model for contrastive learning using Facenet technique("FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff, et.al") and further used the embedding for training a Linear SVM for classification problem. The achieved F1 score is 0.83.

Problem Though the scores of both the concept are closeby, I tried to evaluate the quality of both the embeddings using Silhouette Coefficient.

Overall Silhouette Coefficient:

  • Transfer Learning Embedding = 0.05
  • Contrastive Learning Embedding = 0.49

I do not understand this behaviour of the system that even with lower Silhouette Coefficient, the transfer learning model is able to perform well.

Kindly provide me with your views on it

$\endgroup$
2
  • 1
    $\begingroup$ Can you please be more specific and put your specific question in the title (too)? "Kindly provide me with your views on it" is not a question and it's also not specific and "A vs B" is also not a question. $\endgroup$
    – nbro
    Jun 2 at 22:09
  • $\begingroup$ Thanks a lot @nbro. I was bit unsure how to frame such a long question in the title. But I will try to explain the part which was unclear to you. Thanks once again. $\endgroup$ Jun 2 at 22:41

1 Answer 1

0
$\begingroup$

what are the dimensions of the embeddings? The first thing that comes to mind is that you probably should be wary of using euclidean distance in high dimensions (Distribution of Distances between Elements in a Compact Set), as done when computing the Silhouette Coefficient.

However, that's probably not an issue for typical dimensions of 10 to 100. It would be good to get more details on your training.

$\endgroup$
1
  • $\begingroup$ As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Jun 29 at 1:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .