0
$\begingroup$

I have read that when using VAE-GANs, first what happens is the VAE's encoder encodes some image to another encoded image, which from GAN's point of view is considered a noise, and then the GAN part generates another image from that noise which from VAE's point of view is just an encoded image. Does that enoded image is better suited for GAN to generate better images or not? The problem which bugs me is that there are not that many articles about VAE-GANs especially in the last 2 years. As a side question, does that mean that VAE-GANs do not have any significant performance benefits than just simple GAN?

$\endgroup$
1
  • $\begingroup$ I'm not sure about Variational AE, but take a look at CycleGAN. The generator serves as AE. It takes an image as input and the first part of the network is an encoder followed by 9 ResBlocks. The last part is a decoder. The authors do not use random noise. Instead, features of an image an extracted with an encoder and used to generate a new image. The same can be done for StyleGAN. It is possible to encode a style of an image and pass it to the generator. $\endgroup$ Jun 3 at 14:48
0
$\begingroup$

In my experience, it's not a matter of performance benefits; Variational Auto-Encoder GANs are much more useful if you want to have "knobs" to turn to influence the generated output. Since you have a latent layer that represents possibly the mean and the distribution of the data, you can tune to different "positions" in that latent space to influence the output of the GAN.

Without this, the generated output is more difficult to predict and/or you will end up with some outputs that are great and others that are completely meaningless.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.