Can a face recognition system be trained using only computer generated hyper realistic faces?

Question

In order to train a face recognition system you need to have access to a large database with thousands of photos containing different faces. Companies like facebook and amazon have these databases but most average people do not.

If you don't have access to a sufficiently large dataset with faces, could you use computer generated random faces instead? I'm asking this because computers are becoming better and better in rendering hyper realistic faces. An example is the meetmike digital human showcase video. Another example is the unreal engine project spotlight video.. Lastly you also have websites like https://thispersondoesnotexist.com/ that can generate random faces.

What if you generate a couple of photos of the same computer generated face and you make sure that each photo shows the face in a different setting or from a different angle. Could you then use such photos to train a facial recognition system that can accurately recognize real people?

Answer 1

I am a bit later to this but in short yes you can. The approach of training on synthetic data and transfer that knowledge to real data is called sim2real gap (borrowed from RL) and for computer vision you can now hear synth2real gap.

Microsoft has been working on that for a while (they do not have faces dataset) and it has published some relevant papers on it:

• Fake It Till You Make It
• 3D face reconstruction with dense landmarks

There are also other approaches using GANs for face generation, although realism is boosted GANs suffer from micro glitches in the images that biases the training data. But even with that glitches you can successfully train.

You can also measure the knowledge transfer by computing some metrics on some public face datasets such as WIDER-FACE.