1
$\begingroup$

I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space dimension.

My VAE structure is as follows : Input : 2048 -> 1024 -> 512 -> Latent space dimension -> 512 -> 1024 -> Output : 2048

Here is what I get in terms of relative error when the latent space dimension goes from 2 to 100 : enter image description here

Everything works as expected at the beginning, but the error starts rising up at around 50 and I have no idea why. With a large latent space dimension, the output is orders of magnitude smaller than the input, which explains the relative error of value 1.

Here is the same figure when I run the exact same experiment but with a normal autoencoder this time.enter image description here

This time the results are consistent.

What's wrong with my VAE ?

Improve this question
$\endgroup$
2
  • $\begingroup$ This is a really interesting phenomena, my guess would be the network is unable to learn a continuous representation in the number of epochs you provided for learning. Try upping the number of epochs in the VAE for learning and see if the error drops. $\endgroup$ – Recessive Jan 12 at 4:54
  • $\begingroup$ @Recessive I already tried that. Went from 50 epochs to 100. Nothing changed, had the same exact behavior. $\endgroup$ – tyassine Jan 12 at 7:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.