9
votes
Accepted
Are decision tree learning algorithms deterministic?
Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure?
Generally, yes. Most decision tree learners, like the common ID3 and ...
- 9,037
8
votes
Accepted
Why does nobody use decision trees for visual question answering?
For vision tasks, neural network models almost always include a number of layers that pool and convolute. The convolutions, in particular, are very useful - they can make the model generalize better ...
- 416
5
votes
Accepted
Why isn't my decision tree classifier able to solve the XOR problem properly?
I can reproduce this problem for an even more easily separable dataset:
The ideal tree for it should be as follows:
However, when I run DecisionTreeClassifier ...
4
votes
Accepted
What do the values of the leaves of the decision tree represent?
Decision tree nodes are split bases on the number of data samples, these numbers indicate the number of data samples they are fit to.
In your case samples = 256. It is further split into two nodes of ...
4
votes
How could decision tree learning algorithms cope with imbalanced classes?
Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem.
If you look carefully at ...
- 9,037
4
votes
Accepted
Is the dropout technique specific only to neural networks?
I'm sure you can use dropout in any parameterized model, but I suspect it'll only really be helpful if you have enough parameters/nodes. Also dropout in neural nets has a Bayesian meaning, Yarin Gal ...
- 1,071
3
votes
Accepted
Why is the exponential loss used in this case?
The loss is
$$\mathcal{L}=\sum_{i=1}^{N} \ell\left(y_{i}, f\left(\mathbf{x}_{i}\right)\right) \equiv \sum_{i=1}^{N} \exp \left(-y_{i} f\left(\mathbf{x}_{i}\right)\right),$$
which can also be written ...
- 37k
3
votes
Accepted
Why are decision trees and random forests scale invariant?
Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number.
Imagine: For a feature that goes from 0 to 100 a cut at 50 may ...
- 256
3
votes
Accepted
How does the decision tree implicitly do feature selection?
Consider a dataset $S \in \mathbb{R}^{N \times (M + 1)}$ with $N$ observations (or examples), where each observation $S_i \in \mathbb{R}^{M + 1}$ is composed of $M$ elements, one value for each of the ...
- 37k
3
votes
Accepted
What makes a machine learning algorithm a low variance one or a high variance one?
What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate ...
- 146
3
votes
Accepted
Why was Go a harder game for an AI to master than Chess?
The branching factor is important, as it limits the effectiveness of search.
However, the branching factor in chess is already too high to effectively search without techniques that reduce the size ...
- 26.5k
3
votes
Decision tree: more than 2 classes, how to represent elements that are in a class vs ones that aren't?
I don't think that is possible with a decision tree, unless there is some measure of confidence that you can use as a threshold.
I ran into the same problem with the ID3 algorithm. It assigns classes,...
- 5,252
3
votes
3
votes
Accepted
Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?
Yes, if you can assume that your data is separable on the features given then ID3 will find a decision tree for it (Note: this will not necessarily be an optimal tree, or even a good tree). To ...
- 577
2
votes
Accepted
What is the intuition behind the entropy formula used in the ID3 algorithm?
Please, take a look at Understanding Shannon's Entropy metric for Information.
The answer for the minus sign is in section 6. The probability logs are less than or equal to $0$, so the minus sign ...
- 176
2
votes
Why are tree-based models more widely used in Medical Diagnosis?
One possible reason may have something to do with the scrutability of models, as described in the first few paragraphs of this article. It presents a case study of a hospital whose policy was to send ...
- 53
2
votes
Accepted
What are possible functions assigned on decision nodes for decision tree prediction?
For a binary split, there are only three possible operations (or arguably only two if you consider one-hot encoding). Any other kind of split would simply not be binary. Almost every tree-based model ...
- 146
2
votes
At which point we have to stop post pruning in decision tree?
There are a variety of conditions we can use when deciding whether to prune a sub-tree or not after generating a decision tree model. There are three common approaches.
We can prune branches with ...
- 9,037
2
votes
Why are decision trees and random forests scale invariant?
Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector ...
- 21
2
votes
Why isn't my decision tree classifier able to solve the XOR problem properly?
The algorithm fails because it is greedy. This means that it takes the first split decision immediately, without taking into account what will happen in next steps.
An alternative would be given by ...
2
votes
What is the most suitable AI technique to use for path planning?
If I had to implement a path exploration/finding algorithm on a robot, I would follow these steps:
Make sure you can detect your position. You need to be able to record your position otherwise you ...
- 361
2
votes
Accepted
What is the difference between Inductive Learning and Connectionist Learning?
All of the statistical learning is about inductive learning.
What is the difference between inductive learning and connectionist learning?
Inductive learning is about identifying patterns from ...
- 806
2
votes
Accepted
Feature extraction timeseries, model compatibility
there's a lot to un-pack in this question.
Why do they only pick 500 rows?
my guess: in order to keep the example running quickly. tsfresh usually takes a while to calculate its features. note that ...
- 211
1
vote
Random forests - are more estimators always better?
I would say that in general situation more estimators are better.
RandomForest fits a lot of estimators - decision trees that take a subset of data (obtained sampling with replacement) and subset of ...
- 2,184
1
vote
Accepted
Mathematical calculation behind decision tree classifier with continuous variables
Of course, it depends on what algorithm you use. Typically, a top-down algorithm is used.
You gather all the training data at the root. The base decision is going to be whatever class you have most of....
1
vote
What are some applications where tree models perform better than neural networks?
Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see here and here), I would say tree-based models e.g. Random Forests, Decision Trees, ...
- 600
1
vote
What is the meaning of test data set in naive bayes classifier or decision trees?
In machine learning, we can use all the datasets as training data in a model. But if there are too many data sets, or too much data, and we do not split them up, our model may be not produce ...
1
vote
What is the meaning of test data set in naive bayes classifier or decision trees?
Your assumption about the test data is not correct completely. Maybe you use the test data to tune your learning algorithm to work better on the test data, but it's not the whole thing. Sometimes you ...
- 1,723
1
vote
“Outside-in” versus “Inside-out” machine learning
It sounds like you are interested in the ideas of intrinsic motivation and attention in the context of machine learning. These are big topics, and the subject of much active research.
Intrinsic ...
- 9,037
1
vote
What makes a machine learning algorithm a low variance one or a high variance one?
An algorithm's bias and variance can be thought of as its property, this can be tweaked with things that we call as hyperparameters, but every algorithm has its own set of assumptions that it makes ...
- 33
Only top scored, non community-wiki answers of a minimum length are eligible