1
$\begingroup$

Can a variational auto-encoder (VAE) learn images whose pixels have been generated from a Gaussian distribution (e.g. $N(0, 1)$), i.e. each pixel is a sample from $N(0, 1)$?

My gut feeling says no, because the VAE adds additional noise $\epsilon$ to the original image in the latent space, and if all images and pixels are random from the same distribution it would be impossible do decode/reconstruct into the particular input image. However, VAEs are a bit of a mystery to me internally. Any help would be appreciated!

Improve this question
$\endgroup$
10
  • $\begingroup$ Hi and welcome to AI SE! The $\epsilon$ that you're talking about probably refers to the sample drawn from $N(0,1)$ during the forward pass of the encoder. This is the so-called "re-parametrization trick", i.e. you do not sample directly from the latent space (i.e. you do not sample directly $z$, i.e. a latent representation of the image), but you sample $\epsilon$, then you combine it with the mean and variance of the distribution of the latent space to produce $z$. Why? Because this apparently makes training more stable (i.e. it's a variance reduction technique). $\endgroup$
    – nbro
    Apr 8, 2020 at 22:05
  • $\begingroup$ @nbro I see, thanks for the explanation! So in this case, would it be that the VAE can actually learn these images composed of random pixels? $\endgroup$
    – user1234544
    Apr 8, 2020 at 22:22
  • $\begingroup$ It's a bit ambiguous what you mean by "images composed of random noise at each pixel (each drawn from, say N(0, 1))"? Do you mean that 1. you add a sample from N(0, 1) to each pixel of the original image (i.e. you add Gaussian noise to some image, e.g. a face of a person), or 2. every pixel is drawn from $N(0, 1)$ (i.e. you generate the images really by drawing a sample from $N(0, 1)$)? I believe it's option two. Anyway, by "random", one usually means draw from the uniform distribution. $\endgroup$
    – nbro
    Apr 8, 2020 at 23:05
  • $\begingroup$ Anyway, I don't think you can learn images whose pixels have been generated from a uniform distribution. In general, in those cases, there will be no pattern to learn (i.e. anything to compress). Have a look at information entropy. However, there are denoising auto-encoders. These can reconstruct the original inputs (i.e. not the noisy ones), but the noisy inputs are "just" noisy (i.e. there are still patterns in those noisy images), so they are not uniformly randomly generated. $\endgroup$
    – nbro
    Apr 8, 2020 at 23:06
  • $\begingroup$ @nbro yes, sorry, it is 2, every pixel is random Gaussian noise. From what you're saying, it can't learn a pattern because there is none? What if we were to use a latent space of size equal to the input space size, would it be able to learn then? $\endgroup$
    – user1234544
    Apr 8, 2020 at 23:36

1 Answer 1

Reset to default
1
$\begingroup$

VAE's try and model the distribution of your data. So it's not going to learn " images composed of random noise at each pixel" per se (though, if overfitting, it could remember them). But it would be very capable of learning the simple noise distribution from which you sampled your random pixels

Improve this answer
$\endgroup$
10
  • $\begingroup$ The question is not well-posed, so you should have waited for clarification before answering. $\endgroup$
    – nbro
    Apr 8, 2020 at 23:09
  • $\begingroup$ I think the question is well posed, you seemed not to have understood it the way I did, is all :) the user seems to wonder if there's something to learn from noise and I believe my answer answered just that $\endgroup$
    – ted
    Apr 8, 2020 at 23:11
  • $\begingroup$ Yes, I admit that I missed one part. But it's still not clear whether the user wants to know about 1. images whose pixels have been generated from $N(0, 1)$ or 2. images to which we have added Gaussian noise. These are two different things! In one case, we are just corrupting the image, in the other case, we are really generating "Gaussian images". $\endgroup$
    – nbro
    Apr 8, 2020 at 23:14
  • $\begingroup$ I really understand the question as being about 1 => "composed of random noise at each pixel" ; but only they can actually answer that, true $\endgroup$
    – ted
    Apr 8, 2020 at 23:25
  • $\begingroup$ Indeed, I meant 1), images are generated where each pixel is Gaussian noise, before anything to do with the VAE. Sorry @nbro for not being clear! $\endgroup$
    – user1234544
    Apr 8, 2020 at 23:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .