Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Say a simple neural network's input is a collection of tags (encoded in some way), and the output is an image that corresponds to those tags. Say this network consists of some dense layers and some reverse (transpose) convolution layers.
What is the disadvantage of this network, that directs people to invent fairly complicated things like GANs or VAEs?
The only disadvantage and difference between these generative models and the method you describe, is the input. You describe inputting tags, where as for a GAN, or VAE, the generation segment of the model takes in some representation of a probability distribution. For a GAN, it's mostly random noise, and for a VAE it is some latent space (see nbros answer).
Your described method prevents the network from properly learning fluid generation. If you have a discrete input, the network will attempt to perform a sort of classification on the input, rather then generation, and so when you try to generate a new image, you will most likely get the image equivalent of gibberish.
In fact, that's why a standard auto-encoder (not variational) doesn't work very well for generation. You would think that you could feed in your own custom input into the latent space:
But if you tried this, you would end up telling the network to try and generate something from a latent space it can't interpret.
Hence where the "variational" part of the VAE comes in. This helps the network learn to generate from a continuous distribution, so no matter what input you use, the network will be able to interpret it and give an appropriate output.
As for a GAN, it is simply fed random noise at each training step, so it too generates based on a continuous distribution.
If you were to try and train your method of generation, I would predict that you would find an average of all images that share similar tags (say you have the tags of cat, dog and brown haired, if you input "dog=1, cat=0, brown haired=1" you would get an average of all brown haired dogs), but if you tried to input a combination of tags the network has not seen, as it has not learnt from a continuous distribution, the resultant image would not be anything like what you'd expect from those tags.
I will only focus on the VAE because I am more familiar with it, but the explanations may also apply to the GAN and other generative models.
In the case of the VAE, you train a neural network not only to generate images but to represent them compactly in a so-called latent space, so you train the VAE to do dimensionality reduction. More precisely, the VAE attempts to learn a probability distribution with smaller dimensionality than the dimensionality of the training data but that hopefully represents the training data. Consequently, the model is forced to learn the essential features of the probability distribution that generated the training data.
The VAE is a generative model rather than a discriminative model. In the case of the neural network that maps inputs to images, you will not be learning a latent probability distribution (unless you formulate your model in such a way), from which you could sample, but you would just be mapping, in a supervised way (that is, you would need a labelled training dataset) and deterministically, the inputs to the images. In the VAE, there's some form of stochasticity, while training (and testing) the model.
