2
$\begingroup$

I'm trying to understand how DeepFakes are generated and so far I understood that they're mostly generated through the usage of GANs and autoencoders.

The autoencoders part is understandable, but what I cannot understand is how to generate faces with GANs that match destination face.

GANs consist of a generator and a discriminator. The generator is getting noise input which is randomly selected from a normal distribution and feedback from the discriminator. The discriminator is taught how the real data looks like and just classifies if the data fed to him is real or fake. Depending on the answer - one of them (generator/discriminator) updates its model. If the discriminator guesses right, the generator is getting updated if not, then the discriminator is the one that is updating its model.

So after the training part is over, we can feed the generator more noise to achieve more fake data. In DeepFake videos, we normally try to swap the destination face with the input face. My problem with that is that the destination face has specific features for example it has closed eyes, smiles, rotates its head. If we feed the generator noise, how can we control the process to achieve similar facial features that are in the destination face?

I've found papers about GANs that can control some of the features of generated faces (StyleGANs). Although I'm not sure how would it be possible to extract "special features" of destination face and generate them with StyleGANs.

I will be extremely grateful for any help in understanding the concept of DeepFake with GANs. Thanks a lot.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.