Multi label classification on non binary labels with pytorch

Question

I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ).

Would be the best approach to hard code all of them or is there some better approach? If this is the best way, does anyone have a similar PyTorch notebook on which I could orientate myself? Or some smart solution how to hard code them automatically?

Any help is welcome!

Answer 1

If your goal is to predict given an image multiple labels (each of them can be binary or multi-class) you could consider two strategies:

• Create for each classification task a separate model, which predicts solves only one problem
• Create a single model with multiple heads

The first option seems to be more straightforward, but it would most likely need to consume more memory and computational resources, and the gradient signal from the prediction of one model is independent of other models. In case the labels are uncorrelated or weakly correlated this is not a problem.

The second option when you have a joint backbone for all classification problems and only, in the end, the computation graph splits into branches solving each of the classification problems seems to be more efficient, and in the case, where these tasks are related to each other, improvement of accuracy in one task is very likely to be beneficial for the other task.

Overall, the resulting architecture resembles the Inception architecture:

You can try to put all classification heads in the very end, or some can disentagle from the other part of network a bit earlier.