1
$\begingroup$

I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.

I was told that batch normalization layers might be trying to find weights that are good for all while training, which is a problem since the distribution of my synthetic data is not exactly equal to the distribution of the real data. So, the idea would be to have different 'copies' of the weights of batch normalization layers. So that the neural network estimates different weights for synthetic and real data, and uses just the weights of real data for evaluation.

My question is, how to perform batch normalization in the aforementioned case? Is there already an implementation of batch norm layers in PyTorch that solves the problem?

$\endgroup$
8
  • $\begingroup$ Does the second part of the question falls in to the category of asking for an API, library, or dataset (to solve a specific problem). which is off-topic? @nbro $\endgroup$
    – hanugm
    Jul 26 at 12:50
  • $\begingroup$ I am interested in first part of the question. $\endgroup$
    – hanugm
    Jul 26 at 13:00
  • $\begingroup$ Oh, in this case? Where should i post this question? $\endgroup$
    – Manveru
    Jul 26 at 13:00
  • $\begingroup$ The question is nice. But i am not sure about the part Is there already an implementation of batch norm layers in PyTorch that solves the problem? only $\endgroup$
    – hanugm
    Jul 26 at 13:01
  • 1
    $\begingroup$ @hanugm If "Is there already an implementation of batch norm layers in PyTorch that solves the problem?" is just a secondary/additional question, i.e. it's not the main question, and by "how to perform batch normalization in the aforementioned case?" the OP is asking, for example, for the pseudocode that would allow them to implement the approach to solve their problem, the question is fine (I think). $\endgroup$
    – nbro
    Jul 26 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.