# Why are Siamese Neural Networks used instead of a single neural network?

Siamese Neural Networks are a type of neural network used to compare two instances and infer if they belong to the same object. They are composed by two parallel identical neural networks, whose output is a vector of features. This vector of features is then used to infer the similarity between the two instances by measuring a distance metric.

I was wondering, why not using instead a single neural network that receives as input the two objects that are being compared (e.g. two images) and directly outputs the similarity score? Wouldn't it be better to let the model compare some features of the intermediate layers? Why the Siamese Neural Networks are used for this task and what are the benefits of a Siamese Neural Network over a single neural network that receives as input two instances (e.g. two images) and directly outputs the distance score?

## 2 Answers

I come up with multiple advantages for siamese against a single neural network for similarity measuring:

Training Phase. If using a single network to replace Siamese, it might be required a double number of parameters (weights) for learning. Hence, training the network will likely converge slower and the network will be more volatile to noise.

Testing Phase. Note that these similarity measurements are used in the applications like face recognition. Now, suppose we are going to use the model in such a system. If we have implemented the model by the Siamese, we would only need to compute the output of the model for the input once, and then use the cached results for the existing images in the database, and eventually fasly compute the similarity measures. On the other hand, if we have implemented the measurement by a single neural network, we should compute per query the result for all combination of the input and images in the background. Hence, in the latter, we cannot cache the results for the existing data in the database. Therefore, single neural network implementation will have much more intensive query time for massin dataset than Siamese implementation.

• Why using a single NN to replace siamese neural networks, which the OP describes as being composed of 2 neural networks, would necessarily double the number of parameters? I am not currently familiar with siamese neural networks, so maybe that's why I have this question. Moreover, regarding your second point, I think I understand what you mean, but note that you could also cache the results of the comparison itself. Of course, for objects that you haven't yet compared, you wouldn't have anything in the database.
– nbro
Apr 16 at 8:06
• @nbro Not necessarily, double. So, I updated the post to might be. For the second part, you're right. We can cache all queries; however, the space complexity of the cache will be the query size that can rapidly grow. But for the Siamese network, it's constant to the number of training data and does depend on the query size. So, for the latter, the cache size is manageable.
– OmG
Apr 16 at 19:03

In addition to @Omg's answer note that Siamese networks are typically used in situations where applying (A,B) to the inputs must generate the same output as applying (B,A) (i.e. the similarity measure of A to B is the same as the similarity of B to A).

With a network with separate weights, this is not guaranteed. One way to get close to this is to not only use samples (A,B) as training input but also (equally often) (B,A). Effectively this doubles the number of training steps (and therefore training time) and the network output is still not guaranteed to be symmetric.

By sharing weights, the symmetry of the response of the network ((A,B) gives the same output as (B,A)) is guaranteed by design.

• You say "With a network with separate weights, this is not guaranteed". But what do you mean by "separate weights"? It seems that the OP is suggesting to use a neural network that receives as input the 2 objects that need to be compared, so the weights of this neural network would, in a way, be shared among these 2 objects, so not separate. So, can you clarify this?
– nbro
Apr 16 at 8:10
• Good point -- OP mentions "why not using instead a single neural network that receives as input the two objects". I implicitly assumed that this would be two subnetworks of identical structure (at least for the first few layers). So effectively I had 'two subnetworks with separate weights' in mind. Apr 17 at 10:43
• Indeed, I was thinking of a single NN in which the input are the two images concatenated. Anyway, at least in the first layer the weights will be different and what Andre Holzner has answered applies. Apr 18 at 15:10