I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the output) are a 12D vector. I tried several sizes for the latent space, but, for some reason, it's not training. From the beginning, the KL loss in almost zero (around 1e-10), while the reconstruction loss (MSE for Gaussian distribution) is around 1, and they basically vary around these values without learning anything further.

Are there any general tips for troubleshooting a VAE (I never trained one before)?

I am pretty sure that the code is right and the data for sure has a background and signal (the ratio is 10:1), so I am not really sure what I am doing wrong.

  • $\begingroup$ Hello bill welcome to AI:SE! Maybe try checking on the gradients to see if they are vanishing? If you ' adjusted the input and output to match what I need' without adjusting the quantity of layers and ensuring that the gradients don't vanish or explode anywhere in the middle then that could be the issue. $\endgroup$ – Michael Hearn Nov 15 '19 at 6:23
  • $\begingroup$ @MichaelHearn thank you for the idea! I will look into it. However, my NN is 6-7 layers deep, so I doubt there is a major gradient problem (I trained a lot deeper ones without BatchNorm or anything special, without any problem). $\endgroup$ – Bill Nov 15 '19 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.