9 votes
Accepted

Are decision tree learning algorithms deterministic?

Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? Generally, yes. Most decision tree learners, like the common ID3 and ...
John Doucette's user avatar
4 votes

How could decision tree learning algorithms cope with imbalanced classes?

Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem. If you look carefully at ...
John Doucette's user avatar
4 votes
Accepted

Why are decision trees and random forests scale invariant?

Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number. Imagine: For a feature that goes from 0 to 100 a cut at 50 may ...
N. Kiefer's user avatar
  • 321
2 votes

Why are decision trees and random forests scale invariant?

Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector ...
Ghostpunk's user avatar
2 votes
Accepted

How should I interpret this validation plot?

This is a sign of overfitting. As you make your trees deeper, it becomes possible to "memorize" the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not ...
John Doucette's user avatar
2 votes

Why are tree-based models more widely used in Medical Diagnosis?

One possible reason may have something to do with the scrutability of models, as described in the first few paragraphs of this article. It presents a case study of a hospital whose policy was to send ...
QuadraticOne's user avatar
2 votes

Oposite type of predictions for unbalanced dataset

There are two main things to consider for dealing with imbalanced data: During Training: Undersampling the majority class (healthy patients) so that the model is not that biased to predicting healthy ...
cookiemonster's user avatar
2 votes

Oposite type of predictions for unbalanced dataset

A random forest is a collection of classification trees. If more than 50% of these trees predict class A (and not class B), the random forest will predict class A. What you can do is lower the ...
Lustwelpintje's user avatar
2 votes

Reinforcing Learning when action has no effect on the environment

Short Intro It's very common for people to think that Deep Learning is a "superior form" of Neural Network, a "smarter model". And then they try to use DL for solving simple tasks ...
Andre Goulart's user avatar
2 votes

Machine Learning Models for Longitudinal Data

I added "longitudinal variables" that take into account the number of times the students took the test and their most recent average cumulative score: My Question: a. Does the approach that ...
J.Kent's user avatar
  • 111
1 vote

How to prevent machine learning to learn misleading correlations

If your model learns correlations using data, then those correlations are necessarily in the data. The only way for the model to not learn those correlations is to change your data. You can either ...
Robin van Hoorn's user avatar
1 vote
Accepted

How do I know if my Random Forest Regressor Model is overfitted?

Hi and welcome to StackExchange! First of all, your dataset is truly, extremely small. Maybe someone can correct me, but I would say 30 points is so small that using RandomForest is not appropriate. ...
Vladimir Belik's user avatar
1 vote

Machine Learning Models for Longitudinal Data

I think there are some things you can do to get it work better. Suggestions: add a column to the input giving number of previous tries at the test. If there was a score to go with it, include the ...
EngrStudent's user avatar
1 vote

Random forests - are more estimators always better?

I would say that in general situation more estimators are better. RandomForest fits a lot of estimators - decision trees that take a subset of data (obtained sampling with replacement) and subset of ...
spiridon_the_sun_rotator's user avatar
1 vote

When do the ensemble methods beat neural networks?

Speed: A classic random forest is O(n) to train and O(1) to run while a feedforward neural network is something like O(n^5) to train and O(n^4) to run, so for many tasks the CART ensemble can train ...
EngrStudent's user avatar
1 vote

What are some applications where tree models perform better than neural networks?

Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see here and here), I would say tree-based models e.g. Random Forests, Decision Trees, ...
Tinu's user avatar
  • 618
1 vote

Can I apply AdaBoost on a random forest?

The random forest (rf) is the perfectly parallel ensemble of CART learners. It uses Gini impurity to inform its split locations, and ensemble summary (mean, mode) to track error. The gradient boosted ...
EngrStudent's user avatar
1 vote
Accepted

How can I determine the bias and variance of a random forrest?

To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods. A great article on this topic that I recommend as a light ...
Krrrl's user avatar
  • 211
1 vote
Accepted

How to interpret this learning curve plot

Note the X index is training set size. For the first and second case, teh training set size starts at 0(or 1). The model will overfit certainly at that data size. When data size increases, the model ...
Clement's user avatar
  • 1,735
1 vote
Accepted

How many trees should be generated in a random forest?

The number of estimators in Random Forest is a hyper-parameter. If you are using SKLearn's Random Classifier you can use one of the following techniques to find a (near) optimal hyperparameter ...
jazib jamil's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible