Is geodesic distance between two similar photos less than the Euclidean distance between them? If so, why?

Question

This is from a ML book:

"Principal component analysis, which we discussed in section 6.3, works when the data lies in a linear subspace. However, this may not hold in many applications. Take, for example, face recognition where a face is represented as a two-dimensional, say 100 × 100 image. In this case, each face is a point in 10, 000 dimensions. Now let us say that we take a series of pictures as a person slowly rotates his or her head from right to left. The sequence of face images we capture follows a trajectory in the 10, 000-dimensional space, and this is not linear. Now consider the faces of many people. The trajectories of all their faces as they rotate their faces define a manifold in the 10, 000-dimensional space, and this is what we want to model. The similarity between two faces cannot simply be written in terms of the sum of the pixel differences, and hence Euclidean distance is not a good metric. It may even be the case that images of two different people with the same pose have smaller Euclidean distance between them than the images of two different poses of the same person. This is not what we want. What should count is the distance along the manifold, which is called the geodesic distance."

Based on this explanation, I have the following question.

Is geodesic distance between two similar photos less than the Euclidean distance between them? If so, why?

Answer 1

Imagine your images are embedded in a space that is a circle. Now, to simplify, we put an extra condition in which each embedding (i.e., an image represented somehow in 2 dimensions) must lie on the circumference of the circle. Then you want to measure the distance between two embeddings, i.e., two distinct points on the space: say $u$ and $v$, as in the picture below.

• The Euclidean distance measures the space between the two points as the length of a line that crosses the inside of the circle.
• Instead, the geodesic distance is defined as the shortest line or path (let's say) that connects the two points by travelling only on the surface (circumference) of the manifold, the circle in this case. Meaning that you can't travel in the inside of that space.

According to our assumptions, we can say that the Euclidean distance is always smaller than the Geodesic, for whatever pair of points $(u,v)\in C$ we pick: this may also include dissimilar images, as you mentioned.

The point is that, in the Euclidean space the notion of similar images is ill formed in the sense that a low distance not necessarily means that the images are semantically close: this is especially true in pixel space, and may still work in practice if the embeddings are designed to be semantically sound in the Euclidean space. The book chapter is trying to say that, for such applications, the use of a distance metric that is able to "reason" on the manifold (i.e., a lower dimensional object embedded in a higher dimensional space), therefore accounting for its shape and curvature, should be a better way to discriminate between semantically similar and dissimilar images. In practice, the Euclidean distance may consider similar two images that have a blue sky but completely different content, while the Geodesic may be more consistent.

(Another problem with the Geodesic is how we compute that. I think that when the manifold's shape is unknown, as often is in ML/DL, the Geodesic is hard to compute or approximate at least, while the Euclidean distance is always cheap.)

Answer 2

In the context mentioned, images live in an n-dimensional space and the manifold is embedded in this space. Because we are in Euclidean space, the Euclidean distance will always be the shortest path between any two points.