9
votes
Accepted
Are decision tree learning algorithms deterministic?
Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure?
Generally, yes. Most decision tree learners, like the common ID3 and ...
- 9,207
4
votes
How could decision tree learning algorithms cope with imbalanced classes?
Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem.
If you look carefully at ...
- 9,207
4
votes
Accepted
Why are decision trees and random forests scale invariant?
Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number.
Imagine: For a feature that goes from 0 to 100 a cut at 50 may ...
- 321
2
votes
Why are decision trees and random forests scale invariant?
Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector ...
- 21
2
votes
Accepted
How should I interpret this validation plot?
This is a sign of overfitting.
As you make your trees deeper, it becomes possible to "memorize" the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not ...
- 9,207
2
votes
Why are tree-based models more widely used in Medical Diagnosis?
One possible reason may have something to do with the scrutability of models, as described in the first few paragraphs of this article. It presents a case study of a hospital whose policy was to send ...
- 53
2
votes
Oposite type of predictions for unbalanced dataset
There are two main things to consider for dealing with imbalanced data:
During Training: Undersampling the majority class (healthy patients) so that the model is not that biased to predicting healthy
...
- 216
2
votes
Oposite type of predictions for unbalanced dataset
A random forest is a collection of classification trees. If more than 50% of these trees predict class A (and not class B), the random forest will predict class A.
What you can do is lower the ...
- 514
2
votes
Reinforcing Learning when action has no effect on the environment
Short Intro
It's very common for people to think that Deep Learning is a "superior form" of Neural Network, a "smarter model". And then they try to use DL for solving simple tasks ...
- 854
2
votes
Machine Learning Models for Longitudinal Data
I added "longitudinal variables" that take into account the number of times the students took the test and their most recent average cumulative score:
My Question:
a. Does the approach that ...
- 111
1
vote
How to prevent machine learning to learn misleading correlations
If your model learns correlations using data, then those correlations are necessarily in the data. The only way for the model to not learn those correlations is to change your data.
You can either ...
- 2,083
1
vote
Accepted
How do I know if my Random Forest Regressor Model is overfitted?
Hi and welcome to StackExchange!
First of all, your dataset is truly, extremely small. Maybe someone can correct me, but I would say 30 points is so small that using RandomForest is not appropriate.
...
- 342
1
vote
Machine Learning Models for Longitudinal Data
I think there are some things you can do to get it work better.
Suggestions:
add a column to the input giving number of previous tries at the test. If there was a score to go with it, include the ...
- 371
1
vote
Random forests - are more estimators always better?
I would say that in general situation more estimators are better.
RandomForest fits a lot of estimators - decision trees that take a subset of data (obtained sampling with replacement) and subset of ...
- 2,564
1
vote
When do the ensemble methods beat neural networks?
Speed:
A classic random forest is O(n) to train and O(1) to run while a feedforward neural network is something like O(n^5) to train and O(n^4) to run, so for many tasks the CART ensemble can train ...
- 371
1
vote
What are some applications where tree models perform better than neural networks?
Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see here and here), I would say tree-based models e.g. Random Forests, Decision Trees, ...
- 618
1
vote
Can I apply AdaBoost on a random forest?
The random forest (rf) is the perfectly parallel ensemble of CART learners. It uses Gini impurity to inform its split locations, and ensemble summary (mean, mode) to track error.
The gradient boosted ...
- 371
1
vote
Accepted
How can I determine the bias and variance of a random forrest?
To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods.
A great article on this topic that I recommend as a light ...
- 211
1
vote
Accepted
How to interpret this learning curve plot
Note the X index is training set size. For the first and second case, teh training set size starts at 0(or 1). The model will overfit certainly at that data size. When data size increases, the model ...
- 1,735
1
vote
Accepted
How many trees should be generated in a random forest?
The number of estimators in Random Forest is a hyper-parameter. If you are using SKLearn's Random Classifier you can use one of the following techniques to find a (near) optimal hyperparameter ...
- 166
Only top scored, non community-wiki answers of a minimum length are eligible