What would be the effect of normalizing each input patch going to Convolutional layer separately.
Let's say our input is 64 channels of the size 224x224 (like is the case for some hidden layers in ResNet). And instead of using Batch Normalization or Instance Normalization, we use "Patch Normalization". Our Convolutional layer is applying 3x3 kernel to the input. So we take each 64x3x3 input slice, calculate it's mean and std, use it to normalize, and then multiply it by the layer weights.
It's rather obvious idea, so I assume it was tested multiple times. But I was unfortunate to find any info about it.
