# Tag Info

Accepted

### Are decision tree learning algorithms deterministic?

Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? Generally, yes. Most decision tree learners, like the common ID3 and ...
• 9,037
Accepted

### Why does nobody use decision trees for visual question answering?

For vision tasks, neural network models almost always include a number of layers that pool and convolute. The convolutions, in particular, are very useful - they can make the model generalize better ...
• 416
Accepted

### Why isn't my decision tree classifier able to solve the XOR problem properly?

I can reproduce this problem for an even more easily separable dataset: The ideal tree for it should be as follows: However, when I run DecisionTreeClassifier ...
Accepted

### What do the values of the leaves of the decision tree represent?

Decision tree nodes are split bases on the number of data samples, these numbers indicate the number of data samples they are fit to. In your case samples = 256. It is further split into two nodes of ...

### How could decision tree learning algorithms cope with imbalanced classes?

Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem. If you look carefully at ...
• 9,037
Accepted

### Is the dropout technique specific only to neural networks?

I'm sure you can use dropout in any parameterized model, but I suspect it'll only really be helpful if you have enough parameters/nodes. Also dropout in neural nets has a Bayesian meaning, Yarin Gal ...
• 1,071
Accepted

### Why is the exponential loss used in this case?

The loss is $$\mathcal{L}=\sum_{i=1}^{N} \ell\left(y_{i}, f\left(\mathbf{x}_{i}\right)\right) \equiv \sum_{i=1}^{N} \exp \left(-y_{i} f\left(\mathbf{x}_{i}\right)\right),$$ which can also be written ...
• 37k
Accepted

### Why are decision trees and random forests scale invariant?

Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number. Imagine: For a feature that goes from 0 to 100 a cut at 50 may ...
• 256
Accepted

### How does the decision tree implicitly do feature selection?

Consider a dataset $S \in \mathbb{R}^{N \times (M + 1)}$ with $N$ observations (or examples), where each observation $S_i \in \mathbb{R}^{M + 1}$ is composed of $M$ elements, one value for each of the ...
• 37k
Accepted

### What makes a machine learning algorithm a low variance one or a high variance one?

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate ...
• 146
Accepted

### Why was Go a harder game for an AI to master than Chess?

The branching factor is important, as it limits the effectiveness of search. However, the branching factor in chess is already too high to effectively search without techniques that reduce the size ...
• 26.5k

### Decision tree: more than 2 classes, how to represent elements that are in a class vs ones that aren't?

I don't think that is possible with a decision tree, unless there is some measure of confidence that you can use as a threshold. I ran into the same problem with the ID3 algorithm. It assigns classes,...
• 5,252

### How to calculate the entropy in the ID3 decision tree algorithm?

Suppose you have data: ...
• 1,343
Accepted

### Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?

Yes, if you can assume that your data is separable on the features given then ID3 will find a decision tree for it (Note: this will not necessarily be an optimal tree, or even a good tree). To ...
Accepted

### What is the intuition behind the entropy formula used in the ID3 algorithm?

Please, take a look at Understanding Shannon's Entropy metric for Information. The answer for the minus sign is in section 6. The probability logs are less than or equal to $0$, so the minus sign ...
• 176

### Why are tree-based models more widely used in Medical Diagnosis?

One possible reason may have something to do with the scrutability of models, as described in the first few paragraphs of this article. It presents a case study of a hospital whose policy was to send ...
Accepted

### What are possible functions assigned on decision nodes for decision tree prediction?

For a binary split, there are only three possible operations (or arguably only two if you consider one-hot encoding). Any other kind of split would simply not be binary. Almost every tree-based model ...

### At which point we have to stop post pruning in decision tree?

There are a variety of conditions we can use when deciding whether to prune a sub-tree or not after generating a decision tree model. There are three common approaches. We can prune branches with ...
• 9,037

### Why are decision trees and random forests scale invariant?

Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector ...

### Why isn't my decision tree classifier able to solve the XOR problem properly?

The algorithm fails because it is greedy. This means that it takes the first split decision immediately, without taking into account what will happen in next steps. An alternative would be given by ...

### What is the most suitable AI technique to use for path planning?

If I had to implement a path exploration/finding algorithm on a robot, I would follow these steps: Make sure you can detect your position. You need to be able to record your position otherwise you ...
• 361
Accepted

### What is the difference between Inductive Learning and Connectionist Learning?

All of the statistical learning is about inductive learning. What is the difference between inductive learning and connectionist learning? Inductive learning is about identifying patterns from ...
Accepted

### Feature extraction timeseries, model compatibility

there's a lot to un-pack in this question. Why do they only pick 500 rows? my guess: in order to keep the example running quickly. tsfresh usually takes a while to calculate its features. note that ...
• 211
1 vote

### Random forests - are more estimators always better?

I would say that in general situation more estimators are better. RandomForest fits a lot of estimators - decision trees that take a subset of data (obtained sampling with replacement) and subset of ...
1 vote
Accepted

### Mathematical calculation behind decision tree classifier with continuous variables

Of course, it depends on what algorithm you use. Typically, a top-down algorithm is used. You gather all the training data at the root. The base decision is going to be whatever class you have most of....
1 vote

### What are some applications where tree models perform better than neural networks?

Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see here and here), I would say tree-based models e.g. Random Forests, Decision Trees, ...
• 600
1 vote

### What is the meaning of test data set in naive bayes classifier or decision trees?

In machine learning, we can use all the datasets as training data in a model. But if there are too many data sets, or too much data, and we do not split them up, our model may be not produce ...
1 vote

### What is the meaning of test data set in naive bayes classifier or decision trees?

Your assumption about the test data is not correct completely. Maybe you use the test data to tune your learning algorithm to work better on the test data, but it's not the whole thing. Sometimes you ...
• 1,723
1 vote

### “Outside-in” versus “Inside-out” machine learning

It sounds like you are interested in the ideas of intrinsic motivation and attention in the context of machine learning. These are big topics, and the subject of much active research. Intrinsic ...
• 9,037
1 vote

### What makes a machine learning algorithm a low variance one or a high variance one?

An algorithm's bias and variance can be thought of as its property, this can be tweaked with things that we call as hyperparameters, but every algorithm has its own set of assumptions that it makes ...

Only top scored, non community-wiki answers of a minimum length are eligible