Why is the ELBO of $p(x)=\int p(x|z)p(z)\mathrm{d}z$ easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that aggregating over data means taking log before summing over, but does it create any complication?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
To calculate $p(x) = \int p(x|z) p(z) dz$, you have to calculate it with all configurations of $p(x|z)$ and $p(z)$, which scales exponentially with time. Thus, it is easier to estimate $p(x)$ than to calculate it correctly.
-
$\begingroup$ I don't think that makes sense. x is a fixed value given by data. I'm basically asking why p(x) can't be estimated through sampling the same way as the ELBO can. Also, I'm not sure what time means in your comment. I'll taking about VAEs in general. $\endgroup$ Commented Mar 5, 2023 at 4:47
-
$\begingroup$ Time here means the time complexity you need to calculate all configurations possible with $\int p(x|z) p(z)$. When $z$ is large, it is computationally exhaustive to calculate the value. We are generally good with estimations. $\endgroup$ Commented Mar 5, 2023 at 5:01
-
$\begingroup$ As I mentioned above, this shouldn't be the reason as ELBO also involves an integral over z. It's an expectation with q(z). $\endgroup$ Commented Mar 5, 2023 at 5:47