Questions tagged [evidence-lower-bound]

For questions about the Evidence Lower BOund (ELBO) objective function, which is typically optimized in the context of variational auto-encoders or variational Bayesian neural networks.

Filter by
Sorted by
Tagged with
1
vote
1answer
41 views

How does implementation of VAE's objective function equate to ELBO?

For a lot of VAE implementations I've seen in code, it's not really obvious to me how it equates to ELBO. $$L(X)=H(Q)-H(Q:P(X,Z))=\sum_ZQ(Z)logP(Z,X)-\sum_ZQ(Z)log(Q(Z))$$ The above is the definition ...
3
votes
1answer
128 views

Why does the variational auto-encoder use the reconstruction loss?

VAE is trained to reduce the following two losses. KL divergence between inferred latent distribution and Gaussian. the reconstruction loss I understand that the first one regularizes VAE to get ...
1
vote
1answer
42 views

In this VAE formula, why do $p$ and $q$ have the same parameters?

In $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i),$$ why does $p(x^1,...,x^N)$ and $q(z|x^i)$ have the same parameter $\theta?$ Given that $p$ is ...
3
votes
1answer
174 views

Why is the evidence equal to the KL divergence plus the loss?

Why is the equation $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$$ true, where $x^i$ are data points and $z$ are latent variables? I was ...
2
votes
0answers
46 views

Why does the ELBO come to a steady state and the latent space shrinks?

I'm trying to train a VAE using a graph dataset. However, my latent space shrinks epoch by epoch. Meanwhile, my ELBO plot comes to a steady state after a few epochs. I tried to play around with ...
4
votes
2answers
263 views

What's going on in the equation of the variational lower bound?

I don't really understand what this equation is saying or what the purpose of the ELBO is. How does it help us find the true posterior distribution?