How to normalize images before training?

Question

I have seen people normalize images by just dividing 255. But why? Why not use mean normalization or Z-score Normalization? I also came across this StackOverflow topic while searching but the answers there were not enough enlightening for me.

Answer 1

tl;dr subtracting the mean and dividing by the standard deviation is theoretically more sound, but is impractical compared to dividing by $255$.

---

Explanation

As you know neural networks perform better when their input is scaled. The 2 most common ways to perform scaling are:

• normalization, where you want to scale the image to the $[0, 1]$ range.
• standardization, where you want to bring the mean to $0$ and the standard deviation to $1$.

Theoretically, I think, the latter has some advantages, however in practice there is not a significant difference, especially if the network has normalization layers.

Images are usually stored in 8-bit color mode, meaning that they take integer values from 0 to 255. Because you know the minimum, and maximum values that each pixel can take, it is extremely easy to normalize an image (i.e. you simply have to divide each pixel value by $255$).

On the other hand it is much less practical to compute the mean and standard deviation of the pixel intensities of the training set. Keep in mind that in most cases the training set doesn't even fit into memory, and you have to approximate the true mean/std by moving averages (i.e. load a few images, compute their mean/std, close them, load some more, add their mean to the MA, ...). But why bother if you can similar results by simply dividing each image by $255$.