I have a three-class classification problem for a large dataset. Classes are 0, 1, and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, it can only belong to either class 1 or class 2. On the other hand, if that categorical variable is negative for another sample point, then this sample can be from all three classes. I was wondering how I can make a neural network use that information during training? I guess I need a custom loss function however I could not figure out how to exactly create that.
I think a custom loss function would be an overkill in this situation. A linear pattern like this would be easily learned by any loss desinged for multi class classification.
If I were you I would try 3 roads, in this order:
- train a classifier based on decision trees (random forest & xgboost in primis). The rule you described would most likely be learned as first split to perform, if the remaining features are easily separable then such classifier would perform better than a neural network, plus it would be interpretable.
- train a neural network without any fancy loss function, again the rule you describe is linear and really simple, no reason to think any loss function would miss it. BUT, if you have specific reasons to think that or if you want to try to get the best of the best out of your data there is still another possibility.
- train an ensambled model. More precisely, split the data yourself into 2 subsets, one with positive feature "x" and one with negative feature x, and train 2 separate models on each subset of training data. It is reasonable to think that the model trained on the subset associated with positive feature x would perform better, since the problem will turn from multi class classification to binary classification, but with pros comes cons as well, specifically higher risk of over fitting or class unbalance depending on the stats of your overall dataset.