Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Let us for these purposes say with are working with any feed forward neural network.
Let us also say, that we know beforehand that certain portion of our dataset arsignificantly more impactful or important to our underlying representation. Is there anyway to add that “weighting” to our data?
Yes, You just weight the entries of loss function like sum( Wi(li-t2)**2) for MSE. Most frameworks provides this ability. For example in keras fit method has a parameter - sample_weight: Optional Numpy array of weights for the training samples.
I think you are looking for "Class Balancing" if I understand your question correctly. In most frameworks you can pass an additional weights tensor/array to the loss function/criterion. This is essentially just a weighting factor for each loss term in the sum of the total loss. Through that you achieve that the error/loss for some classes are weighted higher than other classes. Usually you weight classes higher that are less present in your dataset to emphasize that these are not "ignored" by your network over other classes.
Second what you can do if it's your data samples that are imbalanced and not the trained classes you can of course weight the data sampling of your dataloader. Instead of using a uniform distribution over all data samples you can weight specific samples higher or lower so that they are shown more/less often to the network.
