3
$\begingroup$

I am new to GANs. I noticed that everybody generates a random vector (usually 100 dimensional) from a standard normal distribution $N(0, 1)$. My question is: why? Why don't they sample these vectors from a uniform distribution $U(0, 1)$? Does the standard normal distribution has some properties that other probability distributions don't have?

$\endgroup$
0
$\begingroup$

It has become our human bias that data will arrive from a normal distribution. It is also the most prevalent distribution in nature occurring in many places. Hence, we sample from a normal distribution. Also, central limit theorem works around means lying around normal distribution.

It is not taboo to use others if they are helpful to your network. But, a data from one distribution can transformed into other and if that is something that is required, the network will to do it since it can approximate anything (although a weak assumption, but, hey, its working right?).

$\endgroup$
5
  • $\begingroup$ Sometimes the uniform distribution can be more appropriate. I am not familiar with the details of the GAN, but there's probably a reason for sampling from the Gaussian or maybe this is an assumption or maybe it makes more sense, in this case, to put more density or specific values. Maybe you should explain what that vector that the OP is referring to is used for, cuz, if it is similar to the latent vector in VAEs, then the explanation of sampling from a Gaussian is not just "because we like Gaussians" or "because it's a habit", which is somehow what your answer seems to partially suggest. $\endgroup$
    – nbro
    Mar 17 at 13:47
  • $\begingroup$ It would be great if you could write the answer then. And yes uniform distributions are used. Did I say anywhere that others can't be used or it is taboo? And when I said it occurs in most places, it also occurs in images which GANs are prevalently used for. In a photograph, the subject is the important data, the rest of it is not important. That's a Gaussian for you. $\endgroup$ Mar 18 at 4:07
  • $\begingroup$ You wrote "It has become our human bias that data will arrive from a normal distribution. ... Hence, we sample from a normal distribution". No, it's not a human bias (at least not of all humans). The fact that certain empirical distributions, such as the height of humans, seem to follow a normal distribution does not imply a bias or that we will select a normal distribution because we are biased towards it. $\endgroup$
    – nbro
    Mar 18 at 17:46
  • $\begingroup$ Moreover, you say "Also, central limit theorem works around means lying around normal distribution.", but the CTL is about possibly non-Gaussian distributed r.v.s that collectively follow a Gaussian: how does this relate to the GAN case? $\endgroup$
    – nbro
    Mar 18 at 17:49
  • $\begingroup$ I am giving him the social proof for normal distribution. Even non-Gaussian inadvertently follow the normal distribution somehow. I told you about the distribution of information in images. That is normal. That is enough. $\endgroup$ Mar 18 at 18:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.