Decision trees and random forests may or not be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article Using Random Forest to Learn Imbalanced Data, this Stats SE question and this Medium post. The information across these sources does not seem to be consistent.

How could decision tree learning algorithms cope with imbalanced classes?


Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem.

If you look carefully at the three sources you post, you will find that they actually all agree on this point.

Two of the sources actually propose methods of addressing this shortcoming, by making adjustments to the decision tree learning algorithms. The proposed adjustments are essentially standard solutions to these problems, being applied to decision trees.

An example technique, discussed in the first paper you reference, is changing the weightings of the classes. An inefficient/approximate way to do this is to increase the number of examples from the minority class. For example, if you had an 80/20 split, you could add 3 new copies of each minority class example to move to an 80/80 = 50/50 split. Of course, if you add new data points, your algorithm may take longer to run. Instead, you can just modify the weightings of the classes in your optimization function. This approach is algorithm-specific, and will depend on your loss function, but achieves the same effect, just without needing to increase the number of points you use.

  • 3
    $\begingroup$ I think you should at least mention one specific "adjustment" or solution used to cope with imbalanced datasets, in case people want to look for it ;) $\endgroup$
    – nbro
    Nov 19 '19 at 22:31
  • 1
    $\begingroup$ @nbro Good point, I've added an example adjustment. $\endgroup$ Nov 20 '19 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.