Is normalizing the data a way to improve generalization?

Question

There are many known ways to overcome overfitting or make a model generalize better to unseen data.

Here I would like to ask if normalizing/standardizing/similiraizing the train and test data is a plausible approach.

By similarizing I mean making the images look alike by using some function that could be a Neural Network itself. I know that normally one would approach this the opposite way by augmenting and therefore increasing the variation in the training data. But is also possible to improve the model by restricting the variation of the training and test data?

I know that this may not be the best approach and maybe too complicated but I see some use cases where known techniques of preventing overfitting aren't applicable. In those cases, having a network that can normalize/standardize/similarize the "style" of different images could be very useful.

Unfortunately I didn't find a single paper discussing this approach.

Answer 1

Batch Normalization is usually known to speed up the learning process as it makes the weights in the deeper layers more robust. It restricts the distribution of the weights in a particular layer - this video might tend to be useful to what BatchNorm does. This said, batch normalization does have a regularizing effect which does to tend to increase generalization.

Talking about generalization - A focus on regularization would probably be more helpful