I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space dimension.
My VAE structure is as follows : Input : 2048 -> 1024 -> 512 -> Latent space dimension -> 512 -> 1024 -> Output : 2048
Here is what I get in terms of relative error when the latent space dimension goes from 2 to 100 :
Everything works as expected at the beginning, but the error starts rising up at around 50 and I have no idea why. With a large latent space dimension, the output is orders of magnitude smaller than the input, which explains the relative error of value 1.
Here is the same figure when I run the exact same experiment but with a normal autoencoder this time.
This time the results are consistent.
What's wrong with my VAE ?