Why is my siamese network learning very well in e.g. 1 out of every 5 runs? The rest of the time it's not learning and maintains an accuracy of 0.5.
Any explanations? Is the contrastive loss taken in the embedded space to loose of a constraint?
The task is greyscale signature matching.
Additionally, trying the model on facial matching gives a constant 0.5 accuracy, no learning at all - the images are RGB, and maybe it's a higher-order task in general.
Anyways, would appreciate any and all enlightenment in this matter.
P.S. I'm thinking to try a variational autoencoder for the face dataset, where I then use the trained encoder as the siamese network "head".
I would appreciate any guidance or thoughts on this approach as well.