1
$\begingroup$

Given e.g. 1M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$:

Is it possible to have the bottleneck of the AE network with size 1? In other words, without caring about generalization, is it possible to train a network, where, given only 1 encoded value, it can recreate any of the 1M examples?

$\endgroup$
0
1
$\begingroup$

According to various experimentation on autoencoders, it is very possible to have latent vector of size 1. Various layers can help the downsizing of the original input to a very small size of 1. But an issue may arise during decoding. If you're expecting that through one or two or maybe five layers in decoder you can achieve an accurate reconstruction, it is highly unlikely abd the result will turn out to be blurry. Maybe a great network with various parameters may help the reconstruction without considering generalization as asked by you.

$\endgroup$
2
  • $\begingroup$ Hi and welcome to AI SE! Thanks for trying to contribute. You say "According to various experimentation on autoencoders". Can you cite some work that goes into this direction? $\endgroup$ – nbro May 19 '20 at 11:31
  • $\begingroup$ The answer is completely in competence with my own experimentation. It is not published anywhere I guess. But, I may post a link: towardsdatascience.com/how-to-make-an-autoencoder-2f2d99cd5103 and here, if you tweak the network and latent size, you can have a look into the changes. $\endgroup$ – Anubhav Sachan May 19 '20 at 12:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.