Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Here's a diagram of a variational auto-encoder.
There are 2 nodes before the sample (encoding vector). One is the mean, one is the standard deviation. The mean one is confusing.
Is it the mean of values or is it the mean deviation?
$$\text{mean} = \dfrac{X_i+..+X_n}{N}$$
$$\text{mean deviation} = \dfrac{[X_i|+..+|X_n|}{N}$$
Kind of neither, although leaning towards the first definition of the Mean as a simple average of values.
It's a distribution parameter of the Gaussian, so it's the expected average of samples as the number of samples approaches infinity.
The distrinction is that you could draw three samples, -2, 0 and -1, from a Standard Normal - the mean of samples would be -1, but the distribution mean is still 0.
From a network architecture PoV, it's a learnable transformation applied on top of samples from a Standard Normal, so you only need to learn the transformation and get sampling for 'free'.
