Following the standard setup/notation for a VAE, let $z$ denote the latent variables, $q$ as the encoder, $p$ as the decoder, and $x$ as the label. Let the objective be to maximize the ELBO, where a single sample monte carlo estimate of the ELBO is given by \begin{align*} \log p(x \, | \, z) + \log p(z) - \log q(z \, | \, x) \end{align*}

Now I want to focus only on the $\log p(x \, | \, z)$ term, where $x$ is an image, and the decoder $p$ outputs the mean/variance of a normal distribution for each pixel.

My understanding is that pixel values should be integer 0-255. Now consider a single pixel: suppose that the ground truth for that pixel is the value 10, and the encoder predicts the mean, variance $\mu, \sigma^2$ respectively. Now when computing the ELBO, we have this term \begin{align*} \log p(x \, | \, z) = \log f(10, \mu, \sigma^2) \tag{1} \end{align*} where $f$ is the probability density of the normal distribution. My question is why it is justified to compute $\log p(x \, | \, z)$ using the density considering that image data should be discrete valued. It seems to me that any sampled output between 9.5-10.5 would all get mapped to the correct value of 10. Then it seems that you should take the term in the ELBO as \begin{align*} \log p(x \, | \, z) = \log \big(F(10.5, \mu, \sigma^2) - F(9.5, \mu, \sigma^2)\big) \tag{2} \end{align*} where $F$ is the CDF of the normal distribution.

It seems that all references calculate the ELBO as (1) and none as (2). Why is this justified?

  • $\begingroup$ So you mean you want to use a (discrete) gaussian distribution to model a quantity that has finite support (i.e. {0, 1, .., 255}) ? This doesn't sound like a good design choice. Is there any reference you can provide, or is it your own design ? $\endgroup$
    – ayandas
    Oct 4, 2021 at 11:45
  • $\begingroup$ I'm not sure you understand the question. I am describing the typical setup e.g. proceedings.neurips.cc/paper/2016/file/… or the original paper arxiv.org/abs/1312.6114. the decoder uses a gaussian distribution. The question is asking why it is justified to use the probability density of the normal when considering that the output is an image, which as far as I understand is not technically continuous. $\endgroup$
    – Taw
    Oct 4, 2021 at 15:57
  • $\begingroup$ I also don't understand this question, but maybe because it's been a while I had to do with VAEs. What do you mean by "compute p(x | z) using the probability density corresponding to the value of 10". $\endgroup$
    – nbro
    Oct 4, 2021 at 17:27
  • $\begingroup$ @nbro I have edited the question to be more precise now. $\endgroup$
    – Taw
    Oct 6, 2021 at 2:24
  • $\begingroup$ @Taw Thanks. Now, it's clearer. I've quickly checked the VAE paper and they say that they use a Gaussian and constraint the output to [0, 1] for images. Note that pixels can also been seen as having values in the range [0, 1]. Maybe this partially answers your question. Anyway, I would also advise you to put your specific question in the title now (because right now it's not a question and not specific) $\endgroup$
    – nbro
    Oct 6, 2021 at 12:02


You must log in to answer this question.