In the default implementation of batch normalization (e.g. the one in tensorflow), the training data is normalized according to the mean/variance of the current batch. Then in the so-called inference-mode, it uses the learned mean/variance (i.e. the moving mean/variance computed during training) to normalize the subsequent inputs.
I wonder if we can modify batch normalization in this way: during training, normalize according to the moving average/variance. This variant of 'batch normalization' would be equivalent to how the states/actions are frequently normalized in reinforcement learning.
Note that this behavior cannot be obtained using the tensorflow implementation, as the moving mean/variance are no longer updated in inference mode.