When using neural networks (NNs), we often normalized the inputs. I
think this is done to equally capture the changes in any input
feature, that is, if any feature takes huge values and other features
take small values, we don't want the NN not to be able to "see" the change
in the smaller value.
However, what if we cause the NN to become insensitive to the input,
that is, the NN is not able to identify changes in the input
because the changes are too small?
We don't normalize the input to make the model less sensitive to small changes in the input (theoretically, given the correct optimization strategy, the model will learn to approximate the smaller-ranged input as well).
An example of this would be Convolutional Neural Networks. Traditionally, images were represented with integer values ranging from $0$ to $255$. This means that a given pixel could have only $256$ distinct values. However, assuming we normalize the input, let's say to $[0, 1]$, this gives the pixel a whole range of values to occupy, making the input more sensitive to changes.
Instead, normalization is done to help with the model's convergence.