I'm trying to train a Siamese network to check if two images are similar. My implementation is based on this. I find the Euclidian distance of the feature vectors(the final flattened layer of my CNN) of my two images and train the model using the contrastive loss function.

My question is, how do I get a binary output from the Siamese network for testing (1 if it two images are similar, 0 otherwise). Is it just by thresholding the Euclidian distance to check how similar the images are? If so, how do I go about selecting the threshold? If I wanted to measure the training and validation accuracies, the threshold would have to be increased as the network learns better. Is there a way to learn this threshold for a given dataset?

I would appreciate any leads, thank you.


I am working on something rather similar in the sense that I get a continuous value (between 0 and 1) as output that I want to be binary. I think you should be using the ROC curve (Receiver-Operating-Characteristic) instead of the accuracy. If you want a single value take the area-under-the-curve. The ROC curve is basically true-prositive-rate vs false-positive-rate as you test for different values of threshold. You could also (or instead) use precision vs recall.

Also, the optimal threshold does not depend as much on the dataset as it depends on your neural network. As you said this value will change as your network learns. Simply use the outputted value and test different thresholds.

If do want to keep the accuracy, simply pick the threshold for which is it best (at the given iteration). Keep in mind that you should not define the threshold from the validation set as you would be cheating.

ps: from rereading it seems like you were implying to learn the threshold from inside the network. I believe it should just be a post-process

| improve this answer | |
  • $\begingroup$ I'll look into the ROC curve idea, thank you. more than learning the threshold, I was wanted to know if there are other ways to let a Siamese Network give a 0/1 output or if thresholding the Euclidian distances is the only way $\endgroup$ – Abhijith Dec 10 '18 at 2:12
  • $\begingroup$ Unfortunately I don't know for your question regarding the siamese network. I would think there is no such method. Also note I edited my answer as my description of roc was not accurate. $\endgroup$ – csrev Dec 12 '18 at 19:49

Don't use the Euclidian distance as the similar/dissimilar factor, you'll get better results if you put a couple Dense layers at the top of your Siamese network. You don't mention how large your feature vectors are, but if they are 128D face encodings, this is what I used:

# dist_euclid_layer is a layers.Lambda that performs the euclid distance on the input
first_dense = layers.Dense(512, activation='relu')(dist_euclid_layer)
drop_one = layers.Dropout(0.5)(first_dense)
output_dense = layers.Dense(2, activation='sigmoid')(drop_one)
model = models.Model(inputs=[input_layer], outputs=output_dense)
model.compile(loss=losses.binary_crossentropy, optimizer=opt, metrics=['accuracy'])

In training, use [0,1] or [1,0] for Y to indicate different / similar.

Then in production, use the np.argmax to find the positive match:

predictions_raw = model.predict(vstack_batch_input, batch_size=len(batch_input))
predictions = np.argmax(predictions_raw,axis=1)
tot_matches = np.sum(predictions)
if tot_matches == 1:
    # ... good times, whatever face[N] has prediction[N] == 1 is the matching face
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.