1
$\begingroup$

This is a theoretical question.

Setup

I have a time series classification task in which I should output a classification of 3 classes for every time stamp t.

All data is labeled per frame.

The problem:

In the data set are more than 3 classes [which are also imbalanced].

My net should see all samples sequentially, because it uses that for historical information.
Thus, I can't just eliminate all irrelevant class samples at preprocessing time.

In case of a prediction on a frame which is labeled differently than those 3 classes, I don't care about the result.


My thoughts:

  1. The net will predict for 3 classes
  2. The net will only learn (pass backward gradient) for valid classes, and just not calculate loss for other classes.

Questions

  1. Is this the way to go for "don't care" classes in classification?
  2. How to calculate loss only for relevant classes in Pytorch?
  3. Should I apply some normalization per batch, or change batch norm layers if dropping variable samples per batch?

I am using nn.CrossEntropyLoss() as my criterion, which has only mean or sum as reductions.
I need to mask the batch so that the reduction will only apply for samples whose label is valid.

I could use reduction='none' and do that manually, or I could do that before the loss and keep using reduction='mean'.
Is there some method to do this using built in Pytorth tools?

Maybe this can be done in the data-fetching phase somehow?


I am looking some standard, vanilla, thumb rule implementation to tackle this. The least fancy the better.


I am aware this is more than a single question. They are still not separable, as the solution will be unified most likely.

$\endgroup$
0
$\begingroup$

Don't care classes are not advisable since don't care doesn't have any inherent pattern which would hurt your model inadvertently.

You can assign class weights for your classes. CrossEntropyLoss has an argument 'weights'.

BatchNormalization can be left alone since the normalization should happen on the basis of all data and not just some of the data.

$\endgroup$
6
  • $\begingroup$ So a 0 weight shouldn't be used? How would I solve the problem then? $\endgroup$
    – Gulzar
    Mar 12 at 0:55
  • $\begingroup$ Adding a dummy class I said would be bad. Never said that bout the weight, set 0 or very low, try both. $\endgroup$ Mar 12 at 5:52
  • $\begingroup$ Please explain why a 0 weight is different than a dummy $\endgroup$
    – Gulzar
    Mar 12 at 11:10
  • $\begingroup$ A dummy variable will add an extra class while class weights operate inside the already existing labels. By giving less weights to them they will be factored less in loss calculation. But, if you add dummy class then you will have a new class then the loss will be calculated against (num_classes + 1) classes. Since, your dummy class will have no frequentist patterns in them, it will just confuse your model while minimizing its loss. Also, using class weights is easy. Or you can try out both and see the results. $\endgroup$ Mar 12 at 11:50
  • $\begingroup$ Sorry, I am not following. Please explain how using weights is mathematically different than what I propose - predicting just 3 classes, and not backpropagating for other classes. If I allow predicting for 5 classes, how should I handle those at inference? $\endgroup$
    – Gulzar
    Mar 13 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.