1
$\begingroup$

If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x<X)$ to get a probability greater than 0.

So what is the meaning of probabilities such as $P(x|z)$ in variational autoencoders? I can't think of $P(x|z)$ as meaning $P(x<X|z)$, if $x$ is an image, since $x<X$ don't really make sense (all images smaller than a given one?)

$\endgroup$
0
3
$\begingroup$

Whilst you're right that for any continuous distribution $P(X = x) = 0 \;; \forall x \in \mathcal{X}$ where $\mathcal{X}$ is there support of the distribution, they are not referring to probabilities here, rather they are referring to density functions (though this should really be denoted with a lower case $p$ to avoid confusion such as this).

$p(x|z)$ is a conditional distribution, which is also allowed in the continuous case -- you can also 'mix and match', i.e. $x$ could be continuous and $z$ could be discrete, and vice-versa.

In the paper, all the authors are meaning when they write $p(x|z)$ is the density of $x$ conditioned on $z$; in VAE's with an image application this is the conditional density of the image $x$ given your latent vector $z$.

$\endgroup$
0
$\begingroup$

In VAE's, we want to model the distribution of images $x$ with some latent variable $z$. Because $x$ is a random variable, You can think of $P(x|z)$ as the distribution of images $x$ conditioned on the random variable $z$. So given a particular value of $z$, we can generate a distribution over images $x$.

VAE's try to model images, which are themselves high dimensional 2D data. Given a 28x28 image, we already have 784 latent variables to model. We cannot visualise the distribution over all images $x$. Your notation $P(x < X|z)$ makes sense in a 1D case with a scalar value. However when considering 2D and higher, we have a problem with how we consider what is less then. if $x = (y_1,y_2)$ and $X = (y_3,y_4)$, then is $x < X$ if both $y_1 < y_3$ and $y_2 < y_4$? (I.e all dimensions have to be less than or if only one dimension needs to be less than). When talking about high dimensional space therefore, it is not very useful to denote $P(x < X|z)$ because of difficulty in interpreting the results.

$\endgroup$
1
  • $\begingroup$ I didn't think too much about notation. The point is that P(X), that is, the probability of a single point (in, maybe, a multi dimensional space), in a continuous distribution, is 0. $\endgroup$ – IttayD Apr 15 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.