0
$\begingroup$

So I've built an arcface model with this arcface layer implementation: https://github.com/4uiiurz1/keras-arcface/blob/master/metrics.py

I trained for a few epochs and now that I'm comparing the results I'm quite baffled.

According to the paper and resources I've read the embedding should produce embeddings where similar images are closer and should have a higher cosine similarity.

But I have the exact opposite case, I ran the models embedding layers through the hold out set and to 95% the mismatches are closer than the matches. Thus I have a reversed 95% accuracy.

My feed and labels are correct

I binned similar images in groups similar to here: https://www.kaggle.com/ragnar123/unsupervised-baseline-arcface but for a different dataset.

Could someone guess why this is happening? Is it possible that some initialization would produce the opposite goal?

$\endgroup$

1 Answer 1

1
$\begingroup$

Cosine similarity, $s$, is an angle distance. The nature of its distance is represented in the range $s \in [-1, 1]$ being $s=-1$ very dissimilar and $s=1$ very similar.

However the intuition of distances is that the closest to $0$ it is the more similar. So if you say

if s < threshold: 
    match = True 
else: 
    match = False

It would produce the exact opposite results since cosine similarity is a measure where the closest similarity equals to $1$. So you have 2 options:

  1. Transform cosine similarity into a traditional normalized distance $d \in [0,1]$ where $0$-similar, $1$-dissimilar by transforming: $d = (-x + 1) / 2$. The $-$ sign is a source of hassles in metrics computation.
  2. Do the thresholding with the above equal threshold like threshold=0.8 and then just issue the decission: if s >= threshold: match=True
$\endgroup$
2
  • $\begingroup$ thank you so much! It's been sometime but this intuitively makes sense :-) $\endgroup$
    – user199590
    Apr 27, 2022 at 15:06
  • 1
    $\begingroup$ You are welcome, I have been using it very recently and this issue came along, so glad to help with this :) $\endgroup$
    – JVGD
    Apr 28, 2022 at 7:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .