0
$\begingroup$

A generative adversarial network (GAN) takes a vector of numbers as input and generates an image, based on the input. Each element of the vector causes some feature of the image to change, but the mapping between input and output is not clear, as often happens in deep learning. What is the best way to study the correlation between the vector elements and the output image features? The first approach that comes to mind is to manually change every element and check the result, however I am not sure that this is the best solution.

$\endgroup$
1
$\begingroup$

Let me first provide a brief introduction first

GANs as VAEs are generative models which means they learn exactly what you described: to map a typically small dimensional vector/tensor into a higher dimensional one, which in your case is (interpreted as) an image.

These 2 generative models differ in the actual learning strategy which results typically in making GAN outperform VAE in realistic image generation (but it’s better not to get into the reasons for this here)

However in principle the Generator Network in GAN works in a similar way to the Decoder Network in VAE so as their output space is interpreted as an image, you can interpret their input space as a sort of semantic space hence the values you are setting define the semantic of the output image (in fact you observe the features change)

It is possible to say the generative model learns a Semantic to Appearance Mapping hence it works the opposite way of the classifier model which learns an Appearance to Semantic Mapping

Unfortunately as it typically happens in NN this mapping are definitely hard to understand for humans, nevertheless this is an active area of research

So after this introduction, aimed at giving you an essential understanding of the problem, I can suggest you some papers to get into more details like this one

GAN Dissection: Visualizing and Understanding Generative Adversarial Networks

In case you found it not that clear, I’m planning to write a summary of this paper and publish it on my Medium so you could check it there (I’ll update this answer or add a comment to notify about it)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.