37
votes
Accepted
What is the difference between latent and embedding spaces?
Embedding vs Latent Space
Due to Machine Learning's recent and rapid renaissance, and the fact that it draws from many distinct areas of mathematics, statistics, and computer science, it often has a ...
- 1,031
8
votes
Accepted
Why is the evidence equal to the KL divergence plus the loss?
In variational inference, the original objective is to minimize the Kullback-Leibler divergence between the variational distribution, $q(z \mid x)$, and the posterior, $p(z \mid x) = \frac{p(x, z)}{\...
- 41.1k
5
votes
What is the difference between latent and embedding spaces?
The expression "latent space" explicitly indicates that the space is associated with the mathematical concept of an hidden (or latent) variable, which cannot be observed directly, but only indirectly.
...
- 41.1k
3
votes
How to generate new data given a trained VAE - sample from the learned latent space or from multivariate Gaussian?
Few more clarifications. While the correct thing to do is draw from the prior, we have no guarantees that the aggregated posterior will cover the prior. Think of the aggregated posterior as the ...
- 316
2
votes
Accepted
In this VAE formula, why do $p$ and $q$ have the same parameters?
I will try to answer your questions directly (but I guess I won't be able to), otherwise, this can become quite confusing, given the inconsistencies that can be found across different sources.
In $...
- 41.1k
1
vote
Variational Lower Bound in VAE for Gaussian latent prior
My question is: what exactly is $\log p(x|z,w)$? I don't know what this distribution is exactly, so does one make any assumption about the distribution of x?
I completely understand the question. I ...
- 598
1
vote
Accepted
how the GAN architecture maintain similar images close in the latent space?
The GAN generator is an encoder from a latent space. The latent space is unconstrained by any individual items of training data, it doesn't matter which real images are shown to help train the ...
- 32.9k
1
vote
Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?
I am not sure about the VAE in particular, but the convnext presented here uses the "inverse bottleneck" (i.e. internal representations being higher dimensional than inputs) as one of the ...
- 11
1
vote
Is it possible to have a variable-length latent vector in an autoencoder?
If you use RNNs, then I think the solution is to use padding (zero padding) with max sequence length (that is the max number of words in a text) in order to tell your model to skip the zeros when ...
- 929
1
vote
Accepted
What is meant by degrees of freedom of latent variables?
A good example is the degree of freedom in Student's distribution:
The degrees of freedom refers to the number of independent observations in a set of data.
For example:
When estimating a mean ...
- 1,826
1
vote
Accepted
What are some new deep learning models for learning latent representation of data?
Here's a link to my answer on CV Stack Exchange, where I have mentioned about latent spaces and some deep learning models that learn these representations: https://stats.stackexchange.com/questions/...
- 339
1
vote
What is the difference between latent and embedding spaces?
To give a statistician's answer, the distinction is empirical (embedding) versus theoretical (latent positions). You define a statistical model which has latent positions that you could then try to ...
- 11
Only top scored, non community-wiki answers of a minimum length are eligible