Why different images of the same person, under some restrictions, are in a 50 dimension manifold?

Question

In this lecture (starting from 1:31:00) the professor says that the set of all images of a person lives in a low dimensional surface (compared the the set of all possible images). And he says that the dimension of that surface is 50 and that they get this number by adding the three translations of the body, the three rotations of the head and the independent movements of the face's muscles. He also adds that it may be more than 50 but less than 100. How do we get the number 50 ?

The professor previously said (in the same lecture, 1:29:00) that the set of all the images that we could describe as natural and that we could interpret are in a manifold. I try to understand how the number 50 came up like the following: let's take an image of a person, since it's "natural" then it belongs to that manifold. Hence there is an open set to which this image belongs to and there is a homeomorphic map from this open set to an euclidean space. Let's suppose (I don't know why but it's the only possible thing I could come up with to understand) that all the images of that same person, regardless of his position and expressions..., are in that open space then through the homeomorphic mapping we have the "same points" in an euclidean space, do we get the base of it by decomposing all the possible movements of the person?

I hope someone can clarify things for me, it seems this doesn't only work with images but all types of non-structured types of data.

Answer 1

The number 50 is essentially just a guess based on results when compressing and/or generating data of a certain type. The variables such as "the three translations of the body, the three rotations of the head and the independent movements of the face's muscles" are examples only. There is no known formal map with well-defined parameters that defines a well understood manifold of "clear images of this person" in natural images. The lecturer has not constructed such a map as far as I can tell, but has done some related experiments.

Experimentally, it is possible to establish parameter vectors that work, with models like Variational Autoencoders and Generative Adversarial Networks. Depending on the size of the target image, and amount of variation in subject matter that you want to allow for (pose, lighting, clothing, hair style, makeup, camera properties etc), you will end up with different sizes of embedding vectors that appear to capture the important variations. When dealing with multiple people, it is common to see vector sizes of 64, 128, 256.

The lecture suggests compressing images with a clear background, consistent lighting, same person with only changes being in pose. Around 50 dimensions for this relatively simple image space seems reasonable, given facial recognition engines that work well in a more complex domain using 128 dimensional embeddings.

I expect that the lecturer has seen experimental evidence that a vector of 50 dimensions performs well at representing all variations in these images, plus smaller vectors perform measurably worse and larger vectors do not perform better. This experiment is possible by constructing something like a VAE with a specific size of embedding vector, training it, then measuring loss when reconstructing a set of test images.