Questions tagged [batch-normalization]

Batch normalization is a technique for improving the speed, performance, and stability of artificial neural networks. Batch normalization is used to normalize the input layer by adjusting and scaling the activations.

Filter by
Sorted by
Tagged with
1
vote
0answers
19 views

Does this tutorial use normalization the right way?

There is this video on pythonprogramming.net that trains a network on the MNIST handwriting dataset. At ~9:15, the author explains that the data should be normalized. The normalization is done with ...
1
vote
0answers
23 views

Why does the BatchNormalization layer produce different outputs during training and inference?

I modified resnet50 architecture to get a regression network. I just add batchnorm1d and ReLU layers just before the fully connected layer. During the training, the output of batchnorm1d layer is ...
3
votes
1answer
55 views

What is a “batch” in batch normalization?

I'm working on an example of CNN with the MNIST hand-written numbers dataset. Currently I've got convolution -> pool -> dense -> dense, and for the optimiser I'm using Mini-Batch Gradient Descent with ...
2
votes
1answer
45 views

How does a batch normalization layer work?

I understood that we normalize to input features in order to bring them on the same scale so that weights won't be learned in arbitrary fashion and training would be faster. Then I studied about ...
2
votes
0answers
21 views

How are batch statistics computed in Recurrent Batch Normalization?

I'm implementing recurrent BN per this paper in Keras, but looking at it and those citing it, a detail remains unclear to me: how are batch statistics computed? Authors omit explicit clarification, ...
3
votes
1answer
59 views

Why do current models use multiple normalization layers?

In most current models, the normalization layer is applied after each convolution layer. Many models use the block $\text{convolution} \rightarrow \text{batch normalization} \rightarrow \text{ReLU}$ ...
2
votes
0answers
40 views

How to properly use batch normalization during inference

I am trying to manually implement calculations of the image classification process using pre-trained weights from the MobilenetV2 network. I know how to apply filter weights to the channels, but not ...
3
votes
1answer
43 views

Is batch normalization not suitable for non-gaussian input?

I generate some non-Gaussian data, and use two kinds of DNN models, one with BN and the other without BN. I find that the model DNN with BN can't predict well. The codes is shown as follow: <...
2
votes
1answer
37 views

Is it compulsary to normalize the dataset if doing so can negatively impact a Binary Logistic regression performance?

I am using raw data set with 4 feature variables (Total Cholesterol, Systolic Blood Pressure, Diastolic Blood Pressure, and Cigraeette count) to do a Binominal Classification (find stroke likelihood) ...
2
votes
1answer
53 views

What is the most common practice to apply batch normalization?

For a deep NN, should I generally apply batch normalization after each convolution layer? Or only after some of them? Which? Every 2nd, every 3rd, lowest, highest, etc.?
4
votes
1answer
2k views

Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders?

Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders? I cannot find any resources for that. Is it safe to assume that, since it works for other DNNs, it will also ...