I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ).

Would be the best approach to hard code all of them or is there some better approach? If this is the best way, does anyone have a similar PyTorch notebook on which I could orientate myself? Or some smart solution how to hard code them automatically?

Any help is welcome!

  • $\begingroup$ Is you network supposed to perfrom multilabel-multiclass classification for each training sample? $\endgroup$ Oct 21, 2021 at 15:33
  • $\begingroup$ Hey! Yes, that is the plan. $\endgroup$ Oct 21, 2021 at 18:24

1 Answer 1


If your goal is to predict given an image multiple labels (each of them can be binary or multi-class) you could consider two strategies:

  • Create for each classification task a separate model, which predicts solves only one problem
  • Create a single model with multiple heads

The first option seems to be more straightforward, but it would most likely need to consume more memory and computational resources, and the gradient signal from the prediction of one model is independent of other models. In case the labels are uncorrelated or weakly correlated this is not a problem.

The second option when you have a joint backbone for all classification problems and only, in the end, the computation graph splits into branches solving each of the classification problems seems to be more efficient, and in the case, where these tasks are related to each other, improvement of accuracy in one task is very likely to be beneficial for the other task.

Overall, the resulting architecture resembles the Inception architecture: enter image description here

You can try to put all classification heads in the very end, or some can disentagle from the other part of network a bit earlier.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .