Here's a diagram of a variational auto-encoder.

enter image description here

There are 2 nodes before the sample (encoding vector). One is the mean, one is the standard deviation. The mean one is confusing.

Is it the mean of values or is it the mean deviation?

$$\text{mean} = \dfrac{X_i+..+X_n}{N}$$

$$\text{mean deviation} = \dfrac{[X_i|+..+|X_n|}{N}$$

  • 2
    $\begingroup$ Hello, I'm no expert but I assume is the Mean of the values. Usually you need the mean and the standard deviation to describe a distribution. $\endgroup$ – razvanc92 Feb 4 '20 at 9:19

Kind of neither, although leaning towards the first definition of the Mean as a simple average of values.

It's a distribution parameter of the Gaussian, so it's the expected average of samples as the number of samples approaches infinity.

The distrinction is that you could draw three samples, -2, 0 and -1, from a Standard Normal - the mean of samples would be -1, but the distribution mean is still 0.

From a network architecture PoV, it's a learnable transformation applied on top of samples from a Standard Normal, so you only need to learn the transformation and get sampling for 'free'.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.