Question regarding the ELBO decomposition proposed by Hoffman&Johnson

Question

recently I'm trying to read a paper by Hoffman and Johnson discussing an alternative decomposition of ELBO in variational autoencoders. In formula (9) and (10) of their original paper, they proposed some pre-requisite assumptions as follows:

\begin{equation} q(n, z) \triangleq q(n)q(z|n);\ q(z|n) \triangleq q(z|x_n);\ q(n)=1/N \end{equation}

\begin{equation} p(n, z) \triangleq p(n)p(z|n);\ p(z|n) \triangleq p(z);\ p(n)=1/N \end{equation}

where n is the index of a randomly picked instance from a size $N$ subsample from the data distribution $p(x)$. With these preliminaries, H&J derived equation (11):

\begin{equation} \frac{1}{N}\sum_{n=1}^{N} \mathrm{KL}(q(z_n|x_n)\|p(z_n)) = \mathrm{KL}(q(z)|p(z)) + \log{N} - \mathbb{E}_{q(z)}[\mathbb{H}[q(n|z)]] \end{equation}

I had no problem deriving (11) from (9) and (10), but why $p(z|n)\triangleq p(z)$? Pardon me if I have missed any critical statement in the context, but these authors seemed not to have given any justification on this formula before using it. How could one assume that the posterior probability $p(z|n)$ to get a specific $z$ given an $x_n$ is independent of $x_n$?

I will really appreciate it if you can provide some better insight! Thank you!

Answer 1

As for your question why $p(z|n)\triangleq p(z)$, it's the same reason for the latent prior ${\displaystyle p_{\theta }(z)}$ of traditional VAE which belongs to the decoder as a regularizer and is independent of any input data $x_n$ and is often assumed to be the standard multivariate Gaussian for continuously valued input data such as images mainly due to analytic simplicity, sampling convenience, and noninformative continuous latent space prior. Note the encoder's latent posterior $q(z|n)$ is different and is dependent of the input $x_n$ as you quoted.