# Why does variational auto-encoder use the reconstruction loss?

VAE is trained to reduce the following two losses.

1. KL divergence between inferred latent distribution and Gaussian.

2. the reconstruction loss

I understand that the first one regularizes VAE to get structured latent space. But why and how does the second loss help VAE to work?

During the training of the VAE, we first feed an image to the encoder. Then, the encoder infers mean and variance. After that, we sample $$z$$ from the inferred distribution. Finally, the decoder gets the sampled $$z$$ and generates image. So, in this way, the VAE is trained to make the generated image to be equal with the original input image.

Here, I cannot understand why the sampled $$z$$ should make the original image, since the $$z$$ is sampled, it seems that the $$z$$ does not have any relationship between the original image.

But, as you know, VAE works well. So I think I miss something important or understand it in a totally wrong way.

• see answer here ai.stackexchange.com/questions/18390/… – Gerry P Mar 26 at 12:26
Here, I cannot understand why the sampled $$z$$ should make the original image, since the $$z$$ is sampled, it seems that the $$z$$ does not have any relationship between the original image.
The relationship is that you will be maximizing the ELBO, which implies (and you can see this implication only if you are familiar with the ELBO loss) you will be minimizing the KL divergence between your posterior and the prior to generate the samples $$z$$ (i.e. minimizing because there will be a minus in front of the KL term in the ELBO loss) and maximizing the probability of the reconstructed input. More precisely, $$z$$ is used to reconstruct the input (i.e. the decoder does this), which is then used to calculate the reconstruction loss.
In the mathematical formulations, you will see that the likelihood term of the ELBO is $$p(x \mid z)$$, i.e. the likelihood of the input $$x$$ given $$z$$. The $$z$$ is the input to the decoder, which produces a reconstruction of $$x$$. In practice, people will e.g. use the cross-entropy to then calculate the "reconstruction loss" (e.g. see this PyTorch implementation), which should correspond to this likelihood term $$p(x \mid z)$$. Why does the cross-entropy correspond to a likelihood? Because you can actually prove that the cross-entropy is equivalent to the negative log-likelihood. (Also, note that, in the ELBO loss, $$p(x \mid z)$$ does not appear, but the logarithm of $$p(x \mid z)$$ appears, but, for simplicity, I have used $$p(x \mid z)$$ rather than $$\log p(x \mid z)$$ above.)