Are Autoencoders for noise-reduction only suited to deal with salt-and-pepper kind of noise?

Question

I'm currently looking at NN to deal with noisy data. I like the Autoencoder approach https://medium.com/@aliaksei.mikhailiuk/unsupervised-learning-for-data-interpolation-e259cf5dc957 because it seems to be adaptive and does not require to be trained on specific training data.

However, as it is described in this article it seems to rely on having none-noise samples in the input data that are true to the ground truth, so I wonder if an autoencoder also could work in the case of white or blue noise instead of salt-and-pepper noise?

Answer 1

The autoencoder proposed in the link is similar to a Denoising autoencoder (DAE), in sense that both starting from a noisy image try to reconstruct the original image. The difference is that the noisy pixel are ignored during the backpropagation. The DAE takes in input an image, where before the forward phase noise is applied on the image. On output the DAE have to reconstruct the image without noise.

So, given an image $I$ and $\epsilon \sim \mathcal{N}(0,\,\sigma^{2})$ the noise is applied to the image obtaining the image $\hat{I}$. Now, $\hat{I}$ is the DAE input and the autoencoder is trained to minimize the function $L(I,g(f(\hat{I})))$. Where $f$ is the encoder and $g$ is the decoder.

The image on the left is an image of the MNIST dataset where $\epsilon \sim \mathcal{N}(0,\,0.3)$ and on the right there is the reconstruction of a deep DAE. You can change the type of noise in the image, the results should be the same. So the answer to your question is no, autoencoder are suited also for other type of noise.

If interested you can read the paper "What Regularized Auto-Encoders Learn from the Data-Generating Distribution" Guillaume Alain, Yoshua Bengio where is shown that the DAE is learning an approximation of the gradient of the data distribution.