2022 Developer Survey is open! Take survey.
8 votes
Accepted

Are decision tree learning algorithms deterministic?

Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? Generally, yes. Most decision tree learners, like the common ID3 and ...
user avatar
4 votes

How could decision tree learning algorithms cope with imbalanced classes?

Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem. If you look carefully at ...
user avatar
2 votes

Why are decision trees and random forests scale invariant?

Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector ...
user avatar
2 votes
Accepted

Why are decision trees and random forests scale invariant?

Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number. Imagine: For a feature that goes from 0 to 100 a cut at 50 may ...
user avatar
  • 246
2 votes

Oposite type of predictions for unbalanced dataset

There are two main things to consider for dealing with imbalanced data: During Training: Undersampling the majority class (healthy patients) so that the model is not that biased to predicting healthy ...
user avatar
2 votes

Oposite type of predictions for unbalanced dataset

A random forest is a collection of classification trees. If more than 50% of these trees predict class A (and not class B), the random forest will predict class A. What you can do is lower the ...
user avatar
2 votes
Accepted

How should I interpret this validation plot?

This is a sign of overfitting. As you make your trees deeper, it becomes possible to "memorize" the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not ...
user avatar
2 votes

Why are tree-based models more widely used in Medical Diagnosis?

One possible reason may have something to do with the scrutability of models, as described in the first few paragraphs of this article. It presents a case study of a hospital whose policy was to send ...
user avatar
2 votes

Reinforcing Learning when action has no effect on the environment

Short Intro It's very common for people to think that Deep Learning is a "superior form" of Neural Network, a "smarter model". And then they try to use DL for solving simple tasks ...
user avatar
1 vote

Random forests - are more estimators always better?

I would say that in general situation more estimators are better. RandomForest fits a lot of estimators - decision trees that take a subset of data (obtained sampling with replacement) and subset of ...
user avatar
1 vote

What are some applications where tree models perform better than neural networks?

Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see here and here), I would say tree-based models e.g. Random Forests, Decision Trees, ...
user avatar
  • 600
1 vote
Accepted

How can I determine the bias and variance of a random forrest?

To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods. A great article on this topic that I recommend as a light ...
user avatar
  • 211
1 vote
Accepted

How to interpret this learning curve plot

Note the X index is training set size. For the first and second case, teh training set size starts at 0(or 1). The model will overfit certainly at that data size. When data size increases, the model ...
user avatar
  • 1,715
1 vote
Accepted

How many trees should be generated in a random forest?

The number of estimators in Random Forest is a hyper-parameter. If you are using SKLearn's Random Classifier you can use one of the following techniques to find a (near) optimal hyperparameter ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible