To my knowledge, when it comes to stable diffusion, the VQ-VAE the commonly used method. This differs slightly from vanilla VAE which assumes the encoded features to be a normal distribution and the sampled values from the distribution are considered as the VAE's embedding. However, VQ-VAE doesn't make this assumption and instead finds the most similar embedding from the codebook with the encoder's output. Its focus is to find more meaningful representation from the VAE, resulting in more discrete values than those obtained from AE, as it quantizes the range of values into a single value.
Jack123
- 11
- 2