I am new to GANs. I noticed that everybody generates a random vector (usually 100 dimensional) from a standard normal distribution $N(0, 1)$. My question is: why? Why don't they sample these vectors from a uniform distribution $U(0, 1)$? Does the standard normal distribution has some properties that other probability distributions don't have?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Seems to me that the normal distribution in high dimensions is causing problems with heavily under-sampled regions, so I'm wondering the same thing. $\endgroup$– Tim KuipersCommented Feb 19, 2023 at 4:01
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
It has become our human bias that data will arrive from a normal distribution. It is also the most prevalent distribution in nature occurring in many places. Hence, we sample from a normal distribution. Also, central limit theorem works around means lying around normal distribution.
It is not taboo to use others if they are helpful to your network. But, a data from one distribution can transformed into other and if that is something that is required, the network will to do it since it can approximate anything (although a weak assumption, but, hey, its working right?).
-
$\begingroup$ Sometimes the uniform distribution can be more appropriate. I am not familiar with the details of the GAN, but there's probably a reason for sampling from the Gaussian or maybe this is an assumption or maybe it makes more sense, in this case, to put more density or specific values. Maybe you should explain what that vector that the OP is referring to is used for, cuz, if it is similar to the latent vector in VAEs, then the explanation of sampling from a Gaussian is not just "because we like Gaussians" or "because it's a habit", which is somehow what your answer seems to partially suggest. $\endgroup$– nbroCommented Mar 17, 2021 at 13:47
-
$\begingroup$ It would be great if you could write the answer then. And yes uniform distributions are used. Did I say anywhere that others can't be used or it is taboo? And when I said it occurs in most places, it also occurs in images which GANs are prevalently used for. In a photograph, the subject is the important data, the rest of it is not important. That's a Gaussian for you. $\endgroup$ Commented Mar 18, 2021 at 4:07
-
$\begingroup$ You wrote "It has become our human bias that data will arrive from a normal distribution. ... Hence, we sample from a normal distribution". No, it's not a human bias (at least not of all humans). The fact that certain empirical distributions, such as the height of humans, seem to follow a normal distribution does not imply a bias or that we will select a normal distribution because we are biased towards it. $\endgroup$– nbroCommented Mar 18, 2021 at 17:46
-
$\begingroup$ Moreover, you say "Also, central limit theorem works around means lying around normal distribution.", but the CTL is about possibly non-Gaussian distributed r.v.s that collectively follow a Gaussian: how does this relate to the GAN case? $\endgroup$– nbroCommented Mar 18, 2021 at 17:49
-
$\begingroup$ I am giving him the social proof for normal distribution. Even non-Gaussian inadvertently follow the normal distribution somehow. I told you about the distribution of information in images. That is normal. That is enough. $\endgroup$ Commented Mar 18, 2021 at 18:01