0
$\begingroup$

Let us for these purposes say with are working with any feed forward neural network.

Let us also say, that we know beforehand that certain portion of our dataset arsignificantly more impactful or important to our underlying representation. Is there anyway to add that “weighting” to our data?

$\endgroup$
  • 1
    $\begingroup$ Please clarify what algorithm you are using, the answer will vary significantly. $\endgroup$ – Andrew Butler Mar 7 '18 at 4:34
  • 1
    $\begingroup$ You can add a bias to your step function $\endgroup$ – solarflare Mar 7 '18 at 5:43
  • $\begingroup$ You could add some sort of "Experience replay" algorithm to pass the same data several times. $\endgroup$ – Aleksei Maide Mar 7 '18 at 18:28
1
$\begingroup$

Yes, You just weight the entries of loss function like sum( Wi(li-t2)**2) for MSE. Most frameworks provides this ability. For example in keras fit method has a parameter - sample_weight: Optional Numpy array of weights for the training samples.

| improve this answer | |
$\endgroup$
1
$\begingroup$
  • I think you are looking for "Class Balancing" if I understand your question correctly. In most frameworks you can pass an additional weights tensor/array to the loss function/criterion. This is essentially just a weighting factor for each loss term in the sum of the total loss. Through that you achieve that the error/loss for some classes are weighted higher than other classes. Usually you weight classes higher that are less present in your dataset to emphasize that these are not "ignored" by your network over other classes.

  • Second what you can do if it's your data samples that are imbalanced and not the trained classes you can of course weight the data sampling of your dataloader. Instead of using a uniform distribution over all data samples you can weight specific samples higher or lower so that they are shown more/less often to the network.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.