I am training an autoencoder meant to detect anomalies. Initially I scaled my data using a min-max scaler. I realized that this scalar isn't the best because anomalies can cause bias in the scalar. Now I've switched to the Robust Scalar. With this change, my learning mean square error has blown up.
It makes sense that the error might be higher because the data isn't nicely scaled between 0 and 1, but 13 orders of magnitude seems high. I've attempted with and without batch normalization right after the input layer but no difference.
Training with min-max scalar results in a validation MSE of
Training with robust scalar results in a validation MSE of
The autoencoder is structured as
All activations are relu except for the output layer which is linear.