Can a variational auto-encoder (VAE) learn images whose pixels have been generated from a Gaussian distribution (e.g. $N(0, 1)$), i.e. each pixel is a sample from $N(0, 1)$?

My gut feeling says no, because the VAE adds additional noise $\epsilon$ to the original image in the latent space, and if all images and pixels are random from the same distribution it would be impossible do decode/reconstruct into the particular input image. However, VAEs are a bit of a mystery to me internally. Any help would be appreciated!

  • $\begingroup$ Hi and welcome to AI SE! The $\epsilon$ that you're talking about probably refers to the sample drawn from $N(0,1)$ during the forward pass of the encoder. This is the so-called "re-parametrization trick", i.e. you do not sample directly from the latent space (i.e. you do not sample directly $z$, i.e. a latent representation of the image), but you sample $\epsilon$, then you combine it with the mean and variance of the distribution of the latent space to produce $z$. Why? Because this apparently makes training more stable (i.e. it's a variance reduction technique). $\endgroup$ – nbro Apr 8 '20 at 22:05
  • $\begingroup$ @nbro I see, thanks for the explanation! So in this case, would it be that the VAE can actually learn these images composed of random pixels? $\endgroup$ – user1234544 Apr 8 '20 at 22:22
  • $\begingroup$ It's a bit ambiguous what you mean by "images composed of random noise at each pixel (each drawn from, say N(0, 1))"? Do you mean that 1. you add a sample from N(0, 1) to each pixel of the original image (i.e. you add Gaussian noise to some image, e.g. a face of a person), or 2. every pixel is drawn from $N(0, 1)$ (i.e. you generate the images really by drawing a sample from $N(0, 1)$)? I believe it's option two. Anyway, by "random", one usually means draw from the uniform distribution. $\endgroup$ – nbro Apr 8 '20 at 23:05
  • $\begingroup$ Anyway, I don't think you can learn images whose pixels have been generated from a uniform distribution. In general, in those cases, there will be no pattern to learn (i.e. anything to compress). Have a look at information entropy. However, there are denoising auto-encoders. These can reconstruct the original inputs (i.e. not the noisy ones), but the noisy inputs are "just" noisy (i.e. there are still patterns in those noisy images), so they are not uniformly randomly generated. $\endgroup$ – nbro Apr 8 '20 at 23:06
  • $\begingroup$ @nbro yes, sorry, it is 2, every pixel is random Gaussian noise. From what you're saying, it can't learn a pattern because there is none? What if we were to use a latent space of size equal to the input space size, would it be able to learn then? $\endgroup$ – user1234544 Apr 8 '20 at 23:36

VAE's try and model the distribution of your data. So it's not going to learn " images composed of random noise at each pixel" per se (though, if overfitting, it could remember them). But it would be very capable of learning the simple noise distribution from which you sampled your random pixels

  • $\begingroup$ The question is not well-posed, so you should have waited for clarification before answering. $\endgroup$ – nbro Apr 8 '20 at 23:09
  • $\begingroup$ I think the question is well posed, you seemed not to have understood it the way I did, is all :) the user seems to wonder if there's something to learn from noise and I believe my answer answered just that $\endgroup$ – ted Apr 8 '20 at 23:11
  • $\begingroup$ Yes, I admit that I missed one part. But it's still not clear whether the user wants to know about 1. images whose pixels have been generated from $N(0, 1)$ or 2. images to which we have added Gaussian noise. These are two different things! In one case, we are just corrupting the image, in the other case, we are really generating "Gaussian images". $\endgroup$ – nbro Apr 8 '20 at 23:14
  • $\begingroup$ I really understand the question as being about 1 => "composed of random noise at each pixel" ; but only they can actually answer that, true $\endgroup$ – ted Apr 8 '20 at 23:25
  • $\begingroup$ Indeed, I meant 1), images are generated where each pixel is Gaussian noise, before anything to do with the VAE. Sorry @nbro for not being clear! $\endgroup$ – user1234544 Apr 8 '20 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.