I modified resnet50 architecture to get a regression network. I just add batchnorm1d and ReLU layers just before the fully connected layer. During the training, the output of batchnorm1d layer is nearly equal to 3 and this gives good results for training. However, during inference, output of batchnorm1d layer is about 30 so this leads to too low accuracy for test results. In other words, outputs of batchnorm1d layers give very different normalized output during training and inference.

What is the reason for this situation, and how can I solve it? I am using PyTorch.

  • 1
    $\begingroup$ What's your batch size? BN is known to work poorly with small batch sizes. $\endgroup$ – yhenon Jan 30 at 16:31
  • $\begingroup$ batch size is 16 $\endgroup$ – Bedrick Kiq Jan 31 at 6:45
  • $\begingroup$ are you confident you are applying the same pre-processing in training and in testing? $\endgroup$ – yhenon Feb 1 at 14:03
  • $\begingroup$ yes, ı used exactly same data for training and testing. The only difference between them is batch size. $\endgroup$ – Bedrick Kiq Feb 3 at 7:34
  • $\begingroup$ did you shuffle your batches properly? meaning samples of the same class aren't always in the same mini-batch $\endgroup$ – yhenon Feb 3 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.