I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space dimension.
My VAE structure is as follows : Input : 2048 -> 1024 -> 512 -> Latent space dimension -> 512 -> 1024 -> Output : 2048
Everything works as expected at the beginning, but the error starts rising up at around 50 and I have no idea why. With a large latent space dimension, the output is orders of magnitude smaller than the input, which explains the relative error of value 1.
This time the results are consistent.
What's wrong with my VAE ?