# L2 normalization in Facenet paper acting as a Constraint?

In the Facenet paper, Under section 3.2 The authors mention that:

The embedding is represented by f(x) ∈ Rd . It embeds an image x into a d-dimensional Euclidean space. Additionally, we constrain this embedding to live on the d-dimensional hypersphere, i.e. ||f(x)||2 = 1.

I don't quite understand how the above equation holds! As far as I understand L2 Norm is same as euclidean distance but I don't quite understand how this imposes ||f(x)||2 = 1 criteria.

• ||**x**||<sub>2</sub> = 1 is the equation for the surface of a hypersphere. Are you asking for the proof of that, or are you asking how the authors enforce the constraint? This does not describe a measurement process, they are describing in words something they do in the code, such as x = x / np.linalg.norm(x) - so they are enforcing the constraint for the embedding. – Neil Slater Feb 26 '18 at 7:15
• my question is how they enforce the constraint – Abhijit Balaji Feb 26 '18 at 8:01

The constraint is enforced with bespoke code. If FaceNet was implemented in NumPy, and the embedding layer vector (pre-constraint) was in the NumPy array h, then the code might look like:

e = h / np.linalg.norm(h)


The variable e would then contain the desired embedding with L2 distance of 1. In practice, even with NumPy, the code might be more complex due to handling mini-batches. More likely this will not be implemented in NumPy.

In this Keras/TensorFlow-based FaceNet implementation you can see how it may be done in practice:

# L2 normalization
X = Lambda(lambda  x: K.l2_normalize(x,axis=1))(X)


This scaling transformation is considered part of the neural network code (it is part of the Keras model building routine in the above snippet), so there needs to be corresponding support for back propagation through the embedding. When using an automatic differentiation framework such as Theano or TensorFlow, then no extra code is required.