Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
In section 3 paragraph 2 of Batch Normalization: Accelerating Deep Network Training b y Reducing Internal Covariate Shift paper (https://arxiv.org/abs/1502.03167) they say that normalizing a layer's input may change what it represents, I understand this. But what do they mean in the bolded part?
For instance, normalizing the inputs of a sigmoid would constrain them
to the linear regime of the nonlinearity.