BACKGROUND: I have a real world problem of developing a U-net-like model for segmenting lung tumors in lung CT images.

On the one hand, I can make this a two class problem:

 1. lung tumor
 2. not lung tumor

On the other hand, I can make this a multiclass problem (for example):

 1. lung tumor
 2. lung
 3. trachea
 4. heart
 5. blood vessels
 6. etc.

Ultimately, I'm only interested in getting the lung tumor right. Thus, the "class of interest" (mentioned in the title) is lung tumor. All other classes are not necessary for my analysis, BUT my intuition is telling me that including more classes may improve the model's ability to segment lung tumors. My rationale is that not lung tumor is very heterogeneous category that can be difficult for a model to classify a pixel into, but giving the model a set of more specific categories for commonly occurring structures may be helpful to model learning. I am aware that the multiclass approach results in more manual annotations (i.e., time, fatigue, and costs), but that is not the focus of this question.

SPECIFIC QUESTION: Which approach -- the two class or multiclass -- will yield higher accuracy for lung tumor segmentation on CT images?

REQUIREMENTS: I seek a response that sheds deep insight into this problem. Responses founded on established mathematical principles are acceptable. Responses founded on empirical evidence from credible literature references are acceptable. The evidence need not be specific to lung tumors, but it should be directly relevant to segmentation tasks.

PRIOR RESEARCH: I have scoured the literature for a head-to-head comparison of these two approaches, but have not found anything useful for lung tumor segmentation.

RELEVANCE TO THE AI COMMUNITY: While my problem is related to lung tumor segmentation, I believe the problem type is of general interest to the broader AI community. For example, one may be interested in segmenting peoples' eyes in facial images, and one can ask whether they should have two classes (eyes vs. not eyes) or multiple classes (eyes vs. nose vs lips vs. chin vs....). Prior knowledge about which method is better would help people plan their annotation approach before embarking on a long and expensive annotation journey.

  • $\begingroup$ Does anyone have suggestions on how to improve this post in order to get answers? $\endgroup$ Oct 25, 2022 at 1:37
  • $\begingroup$ On a pure theoretical basis there's no difference between a 2-class problem or a multi-class problem via more perceptrons in the first layer to correctly partition the multi-feature space as a preparation to fully match target outputs. But if some patterns from the problematic heterogeneous lung-tumor class are mostly different and identifiable from your other multi-classes patterns, then intuitively your NN's probability to misclassify one lung-tumor sample would be smaller (more accurate). $\endgroup$
    – cinch
    Nov 23, 2022 at 5:53
  • $\begingroup$ @mohottnad, I am unclear why the first layer needs more perceptrons. Isn't the first layer the input layer, and so shouldn't the number of perceptrons not change whether it is binary or multi-class? $\endgroup$ Nov 24, 2022 at 12:43
  • $\begingroup$ Also, I did not say that the lung tumor class is heterogeneous. Rather, I said that the not lung tumor class would be heterogeneous because it would include everything else (e.g. normal lung, trachea, heart, blood vessels, etc.). Thus the class would be difficult to learn. In a multiclass setting, that "heterogenous" class would be broken down. My guess is that giving these smaller more "homogeneous" classes would be easier to "learn". $\endgroup$ Nov 24, 2022 at 12:49
  • $\begingroup$ I meant first hidden layer in a MLP above to have more perceptrons to try to correctly partition the features space for multi-class. You can search “linear machines” for multiclassifier. Of course you can also try CNNs. If you interested lung tumor class is reliable to differentiate from non lung tumor classes, then intuitively why bother with multiclassifier? $\endgroup$
    – cinch
    Nov 24, 2022 at 19:02

1 Answer 1


Maybe not a satisfactory answer but I hope this will give you at least a slightly new perspective on how to think about binary vs multiclass problems in a machine learning way.

Let's start from the math: Binary classification and multiclass classification are basically the same, i.e. every loss formulated for binary classification is a special case of the same loss generalized to n classes. If you see explicit classes for both in every main machine learning frameworks is just due to specific different implementations for boosting a bit performances. But if we look at the formula of the main loss used for classification, i.e. cross entropy, you'll see that there is no difference at all between 2 and n classes

  • Cros entropy: $$-\sum_{c=1}^{M}y_{o,c}log(p_{o,c})$$
  • Binary cross entropy (the sum between the 2 classes is just expanded): $$−(ylog(p)+(1−y)log(1−p))$$

Moving on to your rationale: Not lung tumor is an heterogeneous class, hence splitting it into specific classes might help the model learning better features for each one of them. I don't agree a lot with this rationale for a simple reason: first, if it's possible to lean good features for subclasses of a macro category then mapping all those classes back to the main class should be trivial to learn for any deep learning architecture. The opposite instead does not hold. This is why despite the same math behind, tasks with less classes are usually easier to tackle. The only AI area I can think about that leverage this kind of rationale is multi task learning, in which similar tasks like classification and object segmentation/detection are combined to leverage more ground truth data. But in this case the advantage is the combination of different losses that help learning hierarchical features.

So what to do: I personally feel that the right angle to look at the problem is investigating the amount of data at your disposal and their labels distribution. If your fine grained labels generate a sort of balanced distribution then I would go for multiclass, if not I would go for binary classification, even if the binary labels distribution is highly skewed, since for this scenario Focal loss (a weighted variation of cross entropy) works just fine.

  • $\begingroup$ thank you for taking the time to provide a response. The fact that the binary and multiclass cross entropy loss functions take on similar forms is well-appreciated. However, just because the functions are of similar form does not necessarily mean that they are going to be equal/similar. $\endgroup$ Nov 26, 2022 at 14:54
  • $\begingroup$ Second, the problem of mapping fine labels (trachea, lung, blood vessels) to coarse labels (not lung tumor) is--I agree with you--easy. But this isn't a mapping problem. This is a learning problem. Coarse labels in a hierarchy may be easier to learn when the fine labels share common features. This is levered by various hierarchical approaches. However, this approach breaks down when the fine labels do not share common features, as we have here. $\endgroup$ Nov 26, 2022 at 14:57
  • $\begingroup$ about the second point I was indeed talking about learning in the answer. I don't agree with your last statement, hierarchical learning is not about shared features, actually learning differentiated features (good clustered embedding vectors) is the main goal in multi task learning for high level tasks. If I want to better detect faces I need to be able to detect different face elements first, like eyes or ears, which have nothing to do with each other. Once the model learn that, that the different feature layers will just learn to fire to the same output nodes related to the face class. $\endgroup$ Nov 28, 2022 at 8:36
  • $\begingroup$ point is that you can also learn directly detect faces without clustered features, i.e. with features layers that detect a bit of everything instead of being specialized. So again, unless your dataset is perfectly balanced I don't see why you should make your life harder going down the multiclass rode. $\endgroup$ Nov 28, 2022 at 8:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .