4
$\begingroup$

I have a big dataset (28354359 rows) that has some blood values as features (11 features) and the label or outcome variable that tells whether a patient has a virus caused by a Neoplasm or not.

The problem with my dataset is that 2% of the patients that are in my dataset have the virus and 98% does not have the virus.

I am mandatory to use the random forest algorithm. While my random forest model has a high accuracy scores 92%, the problem is that more than 90% of the patients that have the virus are predicted that they don’t have the virus.

I want the opposite effect, I want that my random forest is likely to predict more often that a patient has the virus (even if the patient does not have the virus (ideally I don’t want this side effect , but rather this than the opposite)).

The idea behind this is that performing an extra test (via an echo) could not harm the patient that has not the virus, but not testing a patient will have result terrible for the patient.

Does somebody have advice how I could tweak my random forest model for this task?

I my self experimented with the SMOTE transformation and other sampling techniques but maybe you guys have other suggestion.

I also have tried to apply a cutoff function.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

A random forest is a collection of classification trees. If more than 50% of these trees predict class A (and not class B), the random forest will predict class A.

What you can do is lower the percentage needed to classify it as class A (in your case, patient has the virus). This way, you can tell your random forest to predict class A if only 20% (or 10%, or 5%, ...) of the decision trees actually predicts class A.

I don't know what code you are using for the random forest algoritm, but in most you should be able to ask the % of certainty the random forest has of each class.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Isn't this the same as what a cutoff function does? $\endgroup$
    – jennifer ruurs
    Commented Nov 26, 2019 at 14:40
  • $\begingroup$ yes, I must have missed the last line on your question. Did this not help your accuracy? $\endgroup$
    – Lustwelpintje
    Commented Nov 26, 2019 at 14:49
  • $\begingroup$ It did improve the accuracy, but i am interested in more suggestion :) $\endgroup$
    – jennifer ruurs
    Commented Nov 26, 2019 at 15:26
  • $\begingroup$ It is possible that the 11 features are not enough data for the random forest. 92% accuracy is actually pretty low considering you are right 98% of the time if you just say nobody has the virus. $\endgroup$
    – Lustwelpintje
    Commented Nov 26, 2019 at 15:37
  • $\begingroup$ Have you tried instead of increasing the virus examples with SMOTE decreasing the non virus examples? Only use about 4% of your database (all the virus examples (2%), and a low amount of the non virus examples(2%))? $\endgroup$
    – Lustwelpintje
    Commented Nov 26, 2019 at 15:44
2
$\begingroup$

There are two main things to consider for dealing with imbalanced data:

  1. During Training: Undersampling the majority class (healthy patients) so that the model is not that biased to predicting healthy

  2. During Evaluation: Using a suitable metric to try to evaluate your model and try to optimize on when you are fine-tuning your random-forest. For imbalanced data you usually use F1 score but since a high recall (predicting sick more often) is important here, F2 score (or other F-beta score where beta>1) is more suitable https://en.wikipedia.org/wiki/F1_score

You can also check for example https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html for more about how to deal with imbalanced data in general

Improve this answer
$\endgroup$
2
  • $\begingroup$ is there a reason why undersampling would be better than oversampling? $\endgroup$
    – jennifer ruurs
    Commented Nov 28, 2019 at 17:01
  • $\begingroup$ not really, if you don't have memory/resource issues, you can try both and see which (if any) works better $\endgroup$
    – cookiemonster
    Commented Nov 28, 2019 at 17:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .