4
$\begingroup$

I understand that noise $z$ serves as the input to the generator. Noise $z$ is essentially a vector of random numbers, typically from Gaussian distribution with chosen size of like $100$. However, I don't understand how different noise can produce different images.

Sorry if this sounds like stupid question, but I've been experimenting with GANs and noticed that if I keep the noise constant, the generated image remains the same, seems like the numbers in the noise vector are linked to generating specific image, but I haven't come across an explanation for this online.

$\endgroup$

2 Answers 2

6
$\begingroup$

Let's take this apart. GANs stand for Generative Adversarial Networks. Your question is how GANs are generative (the G part of the name). For this we need to understand what they try to achieve.

In very simple terms, the generator in GANs is a mapping from a probability distribution to another. The original distribution is a Gaussian distribution and the final distribution (we hope) is the data distribution.

The generator is parameterized as a neural network (this is the N part) and the way we learn the mapping from the data is through adversarial training (the A part).

Now for your questions, what you need to understand is that the generator has two important properties:

  1. It's deterministic
  2. Has the capacity to learn very complex mappings

The first property explains your observation that "if I keep the noise constant, the generated image remains the same".

The second property is important for the diversity you observe. It allows small differences in the initial noise to map to very different samples from the learned distribution.

But this comes with a caveat. While the generator is able to learn complex mappings, this is by no means guaranteed. Depending on the effectiveness of the training you might end up with a generator that only generates very few different images (an issue known as mode collapse).

Extra note: The reason that we use a Gaussian source distribution stems from the fact that we know how to easily sample from it. Plus normals are also maximum entropy distributions under fixed variance something that could potentially help with the diversity part.

$\endgroup$
2
  • $\begingroup$ Thanks, but could you explain how the generator has the ability to learn very complex mappings ? $\endgroup$
    – user77925
    Mar 12 at 19:41
  • 1
    $\begingroup$ The generator is a neural network and for these models there exists the universal approximation theorem, which means that if they have enough parameters they can learn an arbitrarily complex function (a mapping is essentially a function). Of course this is only theoretical. In practice we have only finite parameters and also the trickiest part is how to learn these, which is where the adversarial training comes into play. $\endgroup$
    – sfotiadis
    Mar 13 at 20:22
3
$\begingroup$

To further address you comment question, it's GAN's generator network's deep learning ability consisting of multiple layers of nonlinear transformations (e.g., convolutional layers, transposed convolutions, batch normalization, and hierarchical architecture gradually upsampling the latent representation to higher and higher resolutions, etc) to be able to gradually learn and transform from a usually relatively low dimensional Gaussian input noise $z$ into a very complex high-dimensional data output directly. Each layer in the generator captures different aspects of the data generation process such as edges, contours, spatial orientation features, textures, grey-scale or RGB brightness and object shapes which can be usually accounted for by your example size of 100, and it learns to map different regions of the latent space to different features in the output data.

On the other hand, as the other answer rightly pointed out GAN's generator is always deterministic unlike the decoder of variational autoencoder, so intuitively mode collapse is a more common and serious problem for GAN's generator which is its potential limitation.

$\endgroup$
4
  • $\begingroup$ Thanks for your help $\endgroup$
    – user77925
    Mar 13 at 11:21
  • $\begingroup$ Maybe you should briefly explain what mode collapse is and why we care about it. $\endgroup$
    – nbro
    Mar 15 at 14:24
  • 1
    $\begingroup$ @nbro see the accepted answer's explanation of mode collapse - "a generator that only generates very few different images (an issue known as mode collapse)" which is good and I don't want to repeat. If you're not satisfied with that, your comment should be placed under the accepted answer which preceded mine in time. $\endgroup$
    – cinch
    Mar 15 at 15:36
  • $\begingroup$ @cinch Yes, no need to repeat indeed. I read your answer before the other. $\endgroup$
    – nbro
    Mar 15 at 15:38

You must log in to answer this question.