Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [decision-trees]

For question involving decision trees in any form of AI.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

How to Implement decision tree on FPGA?

I have a large decision tree (depth 80, decision node ~25000 and leaf node ~25000) trained on sklearn decision tree classifier. I am thinking to implement it on an FPGA board. What would be the best ...
user avatar
  • 19
0 votes
1 answer
42 views

Why don't we wait if there is no patrons, in this decision tree from Russel and Norvig's book?

I'm reading Russel-Norvig's book about artificial intelligence and now at chapter decision tree where this figure is shown: So far I understood it. This decision tree should answer the question if we ...
user avatar
  • 111
0 votes
0 answers
24 views

How to perform Cost-complexity pruning for large decision tree in minimum amount of time?

I am trying to prune a decision tree with Cost-complexity pruning(CCP). My decision tree is very large. There are 151011 nodes in the tree. There are 54179 ccp_alphas. With this huge number of ...
user avatar
  • 19
1 vote
1 answer
52 views

Random forests - are more estimators always better?

I'm learning about more advanced methods of hyperparameter optimization, such as the Bayesian methods in the scikit-optimize package. For those unfamiliar with the ...
user avatar
1 vote
0 answers
15 views

Optimize parametric Log-Likelihood with a Decision Tree

Suppose there are some objects with features, and the target is parametric density estimation. Density estimation is model-based. Parameters are obtained by maximizing log-likelihood. $LL = \sum_{i \...
user avatar
  • 11
0 votes
0 answers
6 views

Implementing post pruning using reduced error pruning in univariate, binary decision trees

I am working on building binary, univariate decision trees (DT) from scratch and now I am implementing post pruning using reduced error pruning method. Currently I am using in-order traversal for post ...
user avatar
2 votes
1 answer
92 views

Why is the exponential loss used in this case?

I am reading the paper Tracking-by-Segmentation With Online Gradient Boosting Decision Tree. In Section 2.1, the paper says Given training examples, $\left\{\left(\mathbf{x}_{i}, y_{i}\right) \mid \...
user avatar
2 votes
1 answer
241 views

Is the dropout technique specific only to neural networks?

In one Udemy course was mentioned that "dropout is unique to neural networks". However, I remember an example of decision trees where nodes that are not participating in the overall result ...
user avatar
2 votes
1 answer
237 views

How does a decision tree split a continuous feature?

Decision trees learn by measuring the quality of a split through some function, apply this to all features and you get the best feature to split on. However, with a continuous feature it becomes ...
user avatar
  • 1,324
1 vote
1 answer
166 views

Find the expected reward in an expectimax-based dice rolling game?

I have this question that I'm kinda stuck on. It's a game scenario in which we set up an expectimax tree. In the game, you have 3 dice with sides 1-4 that you roll at the beginning. Then, depending on ...
user avatar
  • 21
1 vote
2 answers
586 views

What does the depth of a decision tree depend on?

In these notes, we have the following statement The depth of a learned decision tree can be larger than the number of training examples used to create the tree This statement is false, according to ...
user avatar
0 votes
1 answer
135 views

Mathematical calculation behind decision tree classifier with continuous variables

I am working on a binary classification problem having continuous variables (Gene expression Values). My goal is to classify the samples as case or ...
user avatar
2 votes
1 answer
75 views

What are some applications where tree models perform better than neural networks?

Neural networks are known to be generally better modeling techniques as compared to tree-based models (such as decision trees). Are there any exceptions to this?
user avatar
1 vote
0 answers
45 views

Can I apply AdaBoost on a random forest?

I know the random forest is a bagging technique. But what if my random forest overfits on a dataset, so I reduce the depth of the decision tree and now it is underfitting. In this scenario, can I take ...
user avatar
1 vote
2 answers
1k views

Why are decision trees and random forests scale invariant?

Feature scaling, in general, is an important stage in the data preprocessing pipeline. Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature ...
user avatar
2 votes
1 answer
206 views

Why isn't my decision tree classifier able to solve the XOR problem properly?

I was trying to solve an XOR problem, and the dataset seems like the one in the image. I plotted the tree and got this result: As I understand, the tree should have depth 2 and four leaves. The ...
user avatar
2 votes
0 answers
36 views

Why do we use a weighted average of child entropies when we calculate information gain?

In the decision tree algorithm, why do we use a weighted average of child entropies when we calculate information gain? What is wrong about using the arithmetic mean of entropies?
user avatar
  • 21
1 vote
6 answers
84 views

What is the meaning of test data set in naive bayes classifier or decision trees?

What is the benefit of a test data set, especially for naive bayes estimator or decision tree construction? When using a naive bayes classifier the probabilities are a fact. As far as I know there is ...
user avatar
  • 435
2 votes
0 answers
426 views

Why information gain with entropy as impurity function can't be used as a splitting method for Decision Tree Regression?

In Decision Tree Regression, we can use 'Reduction in Variance' or MSE (Mean Squared Errors) as splitting methods. There are methods like Gini Index, Information Gain, Chi-Square for splitting on ...
user avatar
  • 43
2 votes
1 answer
129 views

What is the most suitable AI technique to use for path planning?

I am making a firetruck using Arduino Uno with flame sensors and ultrasonic sensors to detect how to move and where to go. As this is a project for my university, I am asked to implement AI in it for ...
user avatar
  • 21
3 votes
1 answer
787 views

What is the difference between Inductive Learning and Connectionist Learning?

According to what we know about inductive and connectionist learning, what is the difference between them ? For those who do not know about : Inductive Learning, like what we have in decision tree ...
user avatar
2 votes
0 answers
76 views

How can I classify instances into two categories and then into sub-categories, when the number of features is high?

I'm working with a problem where I have a lot of variables for different cases of different users. Depending on the values of the different variables of a concrete user in a concrete case, the ...
user avatar
2 votes
1 answer
55 views

Feature extraction timeseries, model compatibility

I've got a timeseries with sensor data (e.g. accelerometer and gyroscope). I now want to extract the activity out of it (e.g. walking, standing, driving, ...). I Followed this Jupyter Notebook. But ...
user avatar
1 vote
1 answer
307 views

How does the decision tree implicitly do feature selection?

I was talking with an ex-fellow worker and he told me that the decision tree implicitly applies a feature selection. He told me that the most important feature is higher in the tree because of the ...
user avatar
5 votes
1 answer
72 views

How could decision tree learning algorithms cope with imbalanced classes?

Decision trees and random forests may or not be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article Using Random Forest to ...
user avatar
2 votes
1 answer
222 views

“Outside-in” versus “Inside-out” machine learning

A little background... I’ve been on-and-off learning about data science for around a year or so, however, I started thinking about artificial intelligence a few years ago. I have a cursory ...
user avatar
1 vote
1 answer
81 views

Can the C4.5 algorithm learn a GOAP model?

Goal-oriented action planning (GOAP) is a well-known planning technique in computer games. It was introduced to control the non-player characters in the game F.E.A.R. (2005) by creating an abstract ...
user avatar
3 votes
1 answer
47 views

What are possible functions assigned on decision nodes for decision tree prediction?

In Decision Tree or Random Forest, each tree has a collection of decision nodes (in which each node has a threshold value) and a class labels (or regression values). I know that threshold values are ...
user avatar
1 vote
1 answer
147 views

At which point we have to stop post pruning in decision tree?

Post pruning is start from downward discarding subtree and include leaf node performance. so what is the best point or condition of the tree where we have to stop further pruning.
user avatar
10 votes
1 answer
3k views

Are decision tree learning algorithms deterministic?

Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? What about the random forest?
user avatar
  • 203
2 votes
0 answers
56 views

What kind of decision rule algorithm is usable in this situation?

I am trying to write an AI to a game, where there is no real adversary. This means, that only the AI player has choices in which move to perform, his opponent may or may not react to the move the AI ...
user avatar
2 votes
2 answers
302 views

What makes a machine learning algorithm a low variance one or a high variance one?

Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression. Examples of high-variance machine learning algorithms ...
user avatar
  • 328
0 votes
1 answer
60 views

How many trees should be generated in a random forest?

What are ways of determining the number of trees to be generated in a random forest algorithm?
user avatar
  • 305
2 votes
2 answers
95 views

How can I minimize the number of answers that are relevant to a machine learning model?

Problem: We have a fairly big database that is built up by our own users. The way this data is entered is by asking the users 30ish questions that all have around 12 answers (x, a, A, B, C, ..., H). ...
user avatar
1 vote
1 answer
383 views

Why was Go a harder game for an AI to master than Chess?

AI became superior to the best human players in chess around 20 years ago (when the 2nd Deep Blue match concluded). However, it took until 2016 for an AI to beat the Go world chess champion, and this ...
user avatar
4 votes
1 answer
98 views

Decision tree: more than 2 classes, how to represent elements that are in a class vs ones that aren't?

I'm building a decision tree and would like to separate (for example) the elements that are in class 0 from those in classes 1 and 2, case in point: ...
user avatar
2 votes
1 answer
325 views

What do the values of the leaves of the decision tree represent?

This is more of a technical question rather than a practical one. I've exported a decision tree made with python/scikit learn and would like to know what the "value" field of each leaf corresponds to....
user avatar
9 votes
1 answer
291 views

Why does nobody use decision trees for visual question answering?

I'm starting a project that will involve computer vision, visual question answering, and explainability. I am currently choosing what type of algorithm to use for my classifier - a neural network or a ...
user avatar
4 votes
1 answer
2k views

Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?

Given a dataset with no noisy examples (i.e., it is never the case that for 2 examples, the attribute values match but the class value does not), is the training error for the ID3 algorithm is always ...
user avatar
  • 105
1 vote
1 answer
795 views

What is the intuition behind the entropy formula used in the ID3 algorithm?

What is the intuition behind the following entropy formula used in the ID3 algorithm? $$ \text{info}(D) = -\sum_{i=1}^m p_i \log_2(p_i) $$
user avatar
  • 1,223
2 votes
3 answers
145 views

Why are tree-based models more widely used in Medical Diagnosis?

In Chapter 14.4 (p. 664) of the book Pattern Recognition and Machine Learning by Bishop, it is mentioned that tree-based models are more widely used in Medical Diagnosis. Apart from giving better ...
user avatar
3 votes
1 answer
5k views

How to calculate the entropy in the ID3 decision tree algorithm?

Here is the definition of the entropy $$H(S)=-\sum_{x \in X} p(x) \log _{2} p(x)$$ Wikipedia's description of entropy breaks down the formula, but I still don't know how to determine the values of $X$,...
user avatar