Since the encoding is performed by a Variational Autoencoder, the VAE encoder must output some mean
and log variance
that we can use to sample a latent vector of shape (latent_dim,)
. But then how does Stable Diffusion use a vector of shape 64x64x3
?
Reference (https://arxiv.org/pdf/2112.10752.pdf, page 24, table 12):