Strange artifacts in autoencoder outputs

Question

I'm training an autoencoder, that does not downsample images but processes them in the same size. For example, a 256x256 input will always be processed at 256x256 resolution, only the channels increase deeper in the network. This design is due to the next stage for which the model will be used. L2 regularization on activations is added since this is essentially a sparse autoencoder.

However, I get some strange artifacts in the corners when the model converges:

The left image is the input, the image on the right is the output. The artifacts can be seen on the top-left and top-right corners of the output image.

Can someone explain what causes these artifacts, and how they can be fixed?

Answer 1

Assuming that you're using convolutional layers, those artifacts may be related to the boundary conditions used. The convolution kernels have a spatial support of say 3x3 pixels, meaning that the response at a position is a function of the corresponding inputs in a 3x3 neighborhood at that position. If the position is adjacent to a boundary, say at the upper left pixel, then the south-west, west, north-west, north, and north-east pixels are outside the image region.

If you only use so-called "valid" pixels, that is, you only get responses for those positions where the entire convolution kernel is inside the image region, then you're ignoring some information at the "invalid" pixels. On the other hand, we are missing information at those invalid pixels, so an assumption must be made about those missing pixels.

For your application, in the internal layers, where the expected features are gradient-like / oscillating and localized, you may wish to try zero-padding, because that's like an assumption that one doesn't assume new feature information outside the region.

However, at the first layer itself, the convolution is with a natural image (as in your example), so you need to predict the pixel values at the border, so there you might want to experiment with "repeat" or "reflecting"/"mirror" boundary conditions.

Answer 2

That's a classic checkerboard artifact. I would guess you're using a CNN as encoder/decoder architectures, since is well known that convolution layers, especially at the upsampling phase, cause these kind of issues.

The use of L2 regularization is also a potential culprit. Pixelwise losses are also known to lead to noisy images, since they don't allow reconstruction freedom to the model.

To understand why you can check this blog post, in this case gifs are more understandable than a thousand words, but long story short, is mostly related to emphasized overlapping region generated by the combinations of kernel/stride sizes.

Solutions:

• use some more complex but not pixelwise regularization like perceptual loss
• remove ConvTranspose2D layers if you're using any, and move to other upsampling layers like Upsample (with bilinear or cubic interpolation)
• in my experience a nice trick that works pretty well in production when the artifact are only on the edges or corners is to pad the input image (with reflect setting) and then crop back the generated image in the original size. This way the model generate artifacts only in non important areas of the image.