2
$\begingroup$

I know that the notation $\mathcal{N}(\mu, \sigma)$ stands for a normal distribution. But I'm reading the book "An Introduction to Variational Autoencoders" and in it, there is this notation: $$\mathcal{N}(z; 0, I)$$ What does it mean?

picture of the book: picture of the book An Introduction to Variational Autoencoders

$\endgroup$
  • $\begingroup$ To understand the notation you have to be familiar with multivariate probability distributions. Are you familiar with it? $\endgroup$ – DuttaA Aug 23 at 18:27
  • $\begingroup$ random normal variable $z$ with mean 0 and unit variance. $\endgroup$ – Brale Aug 23 at 18:47
  • $\begingroup$ @Brale isn't it the probability of $N(0,I)$ at the point $z$? because the notation of what you said should be something like $Z\sim N(0,I)$ shouldn't it? $\endgroup$ – Peyman Aug 23 at 20:36
  • $\begingroup$ @peyman The probability of a normal distribution at any specific point is 0. You are referring to the density function evaluated at that point (which is not a probability). $\endgroup$ – David Ireland Aug 23 at 21:15
5
$\begingroup$

It means that $z$ has a (multivariate) normal distribution with 0 mean and identity covariance matrix. This essentially means each individual element of the vector $z$ has a standard normal distribution.

| improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ $P(z)$ means the density function of $z$, so when they say $P(z)=...$ they are saying $z$ has a certain distribution which follows on from the equals sign. Note my other comment that for a continuous distribution such as Gaussian, the probability of getting any exact point is 0, that is $\mathbb{P}(Z=z) = 0$ for all $z$. $\endgroup$ – David Ireland Aug 23 at 21:30
  • 2
    $\begingroup$ now I got it. thank you so much. $\endgroup$ – Peyman Aug 23 at 21:36
  • 2
    $\begingroup$ No worries, as I say, it is definitely an abuse of notation. I know some statistics professors that would cringe at the sight of it. $\endgroup$ – David Ireland Aug 23 at 21:42
  • 3
    $\begingroup$ @DuttaA I find that a lot of ML authors abuse notation. The problem here is that you're not defining a distribution over a random variable. I guess you would read it loosely as 'function $P(z)$ is equal to the normal distribution' which makes no sense. Maybe I am just being pedantic as obviously I knew what it should mean, but the correct notation would be that $Z \sim \text{N}(0, \textbf{I})$, i.e. $Z$ is a random variable that has a normal distribution with the given parameters. $\endgroup$ – David Ireland Aug 24 at 9:19
  • 2
    $\begingroup$ @DuttaA No it isn't just when you are sampling data, it is how you should write it in general -- if you take any probability course from a maths or stats department you would see this notation used universally. If you want to make clear that some data has a distribution you write $X \sim \text{'the distribution'}$. If you are doing Bayesian statistics you don't assign a prior over $z$ by saying $p(z) = ...$, you write $z \sim ...$. What they are doing here in VAEs is assigning a Gaussian prior to $z$, thus all you need to write is $z \sim \text{N}(\mu, \sigma)$. $\endgroup$ – David Ireland Aug 24 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.