# variational autoencoder - decoder output for images

Following the standard setup/notation for a VAE, let $$z$$ denote the latent variables, $$q$$ as the encoder, $$p$$ as the decoder, and $$x$$ as the label. Let the objective be to maximize the ELBO, where a single sample monte carlo estimate of the ELBO is given by \begin{align*} \log p(x \, | \, z) + \log p(z) - \log q(z \, | \, x) \end{align*}

Now I want to focus only on the $$\log p(x \, | \, z)$$ term, where $$x$$ is an image, and the decoder $$p$$ outputs the mean/variance of a normal distribution for each pixel.

My understanding is that pixel values should be integer 0-255. Now consider a single pixel: suppose that the ground truth for that pixel is the value 10, and the encoder predicts the mean, variance $$\mu, \sigma^2$$ respectively. Now when computing the ELBO, we have this term \begin{align*} \log p(x \, | \, z) = \log f(10, \mu, \sigma^2) \tag{1} \end{align*} where $$f$$ is the probability density of the normal distribution. My question is why it is justified to compute $$\log p(x \, | \, z)$$ using the density considering that image data should be discrete valued. It seems to me that any sampled output between 9.5-10.5 would all get mapped to the correct value of 10. Then it seems that you should take the term in the ELBO as \begin{align*} \log p(x \, | \, z) = \log \big(F(10.5, \mu, \sigma^2) - F(9.5, \mu, \sigma^2)\big) \tag{2} \end{align*} where $$F$$ is the CDF of the normal distribution.

It seems that all references calculate the ELBO as (1) and none as (2). Why is this justified?

• So you mean you want to use a (discrete) gaussian distribution to model a quantity that has finite support (i.e. {0, 1, .., 255}) ? This doesn't sound like a good design choice. Is there any reference you can provide, or is it your own design ? Oct 4, 2021 at 11:45
• I'm not sure you understand the question. I am describing the typical setup e.g. proceedings.neurips.cc/paper/2016/file/… or the original paper arxiv.org/abs/1312.6114. the decoder uses a gaussian distribution. The question is asking why it is justified to use the probability density of the normal when considering that the output is an image, which as far as I understand is not technically continuous.
– Taw
Oct 4, 2021 at 15:57
• I also don't understand this question, but maybe because it's been a while I had to do with VAEs. What do you mean by "compute p(x | z) using the probability density corresponding to the value of 10".
– nbro
Oct 4, 2021 at 17:27
• @nbro I have edited the question to be more precise now.
– Taw
Oct 6, 2021 at 2:24
• @Taw Thanks. Now, it's clearer. I've quickly checked the VAE paper and they say that they use a Gaussian and constraint the output to [0, 1] for images. Note that pixels can also been seen as having values in the range [0, 1]. Maybe this partially answers your question. Anyway, I would also advise you to put your specific question in the title now (because right now it's not a question and not specific)
– nbro
Oct 6, 2021 at 12:02