Skip to main content

Questions tagged [decision-trees]

For question involving decision trees in any form of AI.

Filter by
Sorted by
Tagged with
10 votes
1 answer
5k views

Are decision tree learning algorithms deterministic?

Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? What about the random forest?
mallea's user avatar
  • 203
9 votes
1 answer
477 views

Why does nobody use decision trees for visual question answering?

I'm starting a project that will involve computer vision, visual question answering, and explainability. I am currently choosing what type of algorithm to use for my classifier - a neural network or a ...
The Impossible Squish's user avatar
5 votes
2 answers
921 views

Why isn't my decision tree classifier able to solve the XOR problem properly?

I was trying to solve an XOR problem, and the dataset seems like the one in the image. I plotted the tree and got this result: As I understand, the tree should have depth 2 and four leaves. The ...
Pedro Paiva's user avatar
5 votes
1 answer
84 views

How could decision tree learning algorithms cope with imbalanced classes?

Decision trees and random forests may or not be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article Using Random Forest to ...
jennifer ruurs's user avatar
4 votes
1 answer
198 views

Decision tree: more than 2 classes, how to represent elements that are in a class vs ones that aren't?

I'm building a decision tree and would like to separate (for example) the elements that are in class 0 from those in classes 1 and 2, case in point: ...
Jessica Chambers's user avatar
4 votes
1 answer
3k views

Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?

Given a dataset with no noisy examples (i.e., it is never the case that for 2 examples, the attribute values match but the class value does not), is the training error for the ID3 algorithm is always ...
jrk's user avatar
  • 105
3 votes
2 answers
497 views

What makes a machine learning algorithm a low variance one or a high variance one?

Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression. Examples of high-variance machine learning algorithms ...
Posi2's user avatar
  • 368
3 votes
1 answer
428 views

Why is the exponential loss used in this case?

I am reading the paper Tracking-by-Segmentation With Online Gradient Boosting Decision Tree. In Section 2.1, the paper says Given training examples, $\left\{\left(\mathbf{x}_{i}, y_{i}\right) \mid \...
Zhang Liao's user avatar
3 votes
2 answers
6k views

How to calculate the entropy in the ID3 decision tree algorithm?

Here is the definition of the entropy $$H(S)=-\sum_{x \in X} p(x) \log _{2} p(x)$$ Wikipedia's description of entropy breaks down the formula, but I still don't know how to determine the values of $X$,...
Z. Reticulan's user avatar
3 votes
1 answer
846 views

What is the difference between Inductive Learning and Connectionist Learning?

According to what we know about inductive and connectionist learning, what is the difference between them ? For those who do not know about : Inductive Learning, like what we have in decision tree ...
Mahdi Amrollahi's user avatar
3 votes
1 answer
325 views

“Outside-in” versus “Inside-out” machine learning

A little background... I’ve been on-and-off learning about data science for around a year or so, however, I started thinking about artificial intelligence a few years ago. I have a cursory ...
SuperCodeBrah's user avatar
3 votes
1 answer
83 views

What are possible functions assigned on decision nodes for decision tree prediction?

In Decision Tree or Random Forest, each tree has a collection of decision nodes (in which each node has a threshold value) and a class labels (or regression values). I know that threshold values are ...
user9414424's user avatar
3 votes
0 answers
103 views

Why do we use a weighted average of child entropies when we calculate information gain?

In the decision tree algorithm, why do we use a weighted average of child entropies when we calculate information gain? What is wrong about using the arithmetic mean of entropies?
Krushe's user avatar
  • 31
2 votes
2 answers
3k views

Why are decision trees and random forests scale invariant?

Feature scaling, in general, is an important stage in the data preprocessing pipeline. Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature ...
stoic-santiago's user avatar
2 votes
1 answer
291 views

Is the dropout technique specific only to neural networks?

In one Udemy course was mentioned that "dropout is unique to neural networks". However, I remember an example of decision trees where nodes that are not participating in the overall result ...
bridgemnc's user avatar
  • 125
2 votes
1 answer
826 views

What do the values of the leaves of the decision tree represent?

This is more of a technical question rather than a practical one. I've exported a decision tree made with python/scikit learn and would like to know what the "value" field of each leaf corresponds to....
Jessica Chambers's user avatar
2 votes
1 answer
1k views

How does the decision tree implicitly do feature selection?

I was talking with an ex-fellow worker and he told me that the decision tree implicitly applies a feature selection. He told me that the most important feature is higher in the tree because of the ...
jennifer ruurs's user avatar
2 votes
1 answer
161 views

What is the most suitable AI technique to use for path planning?

I am making a firetruck using Arduino Uno with flame sensors and ultrasonic sensors to detect how to move and where to go. As this is a project for my university, I am asked to implement AI in it for ...
mushter's user avatar
  • 21
2 votes
1 answer
63 views

Feature extraction timeseries, model compatibility

I've got a timeseries with sensor data (e.g. accelerometer and gyroscope). I now want to extract the activity out of it (e.g. walking, standing, driving, ...). I Followed this Jupyter Notebook. But ...
Wouter Vandenputte's user avatar
2 votes
3 answers
182 views

Why are tree-based models more widely used in Medical Diagnosis?

In Chapter 14.4 (p. 664) of the book Pattern Recognition and Machine Learning by Bishop, it is mentioned that tree-based models are more widely used in Medical Diagnosis. Apart from giving better ...
imflash217's user avatar
2 votes
1 answer
87 views

What are some applications where tree models perform better than neural networks?

Neural networks are known to be generally better modeling techniques as compared to tree-based models (such as decision trees). Are there any exceptions to this?
jaiswati_b's user avatar
2 votes
0 answers
742 views

Why information gain with entropy as impurity function can't be used as a splitting method for Decision Tree Regression?

In Decision Tree Regression, we can use 'Reduction in Variance' or MSE (Mean Squared Errors) as splitting methods. There are methods like Gini Index, Information Gain, Chi-Square for splitting on ...
ka1shi's user avatar
  • 43
2 votes
0 answers
90 views

How can I classify instances into two categories and then into sub-categories, when the number of features is high?

I'm working with a problem where I have a lot of variables for different cases of different users. Depending on the values of the different variables of a concrete user in a concrete case, the ...
notarealgreal's user avatar
2 votes
0 answers
63 views

What kind of decision rule algorithm is usable in this situation?

I am trying to write an AI to a game, where there is no real adversary. This means, that only the AI player has choices in which move to perform, his opponent may or may not react to the move the AI ...
Adam Baranyai's user avatar
2 votes
2 answers
167 views

How can I minimize the number of answers that are relevant to a machine learning model?

Problem: We have a fairly big database that is built up by our own users. The way this data is entered is by asking the users 30ish questions that all have around 12 answers (x, a, A, B, C, ..., H). ...
Yvan Stemmerik's user avatar
2 votes
1 answer
3k views

How does a decision tree split a continuous feature?

Decision trees learn by measuring the quality of a split through some function, apply this to all features and you get the best feature to split on. However, with a continuous feature it becomes ...
Recessive's user avatar
  • 1,396
1 vote
1 answer
690 views

Why was Go a harder game for an AI to master than Chess?

AI became superior to the best human players in chess around 20 years ago (when the 2nd Deep Blue match concluded). However, it took until 2016 for an AI to beat the Go world chess champion, and this ...
Inertial Ignorance's user avatar
1 vote
1 answer
147 views

Does the root node of a Decision Tree change when one data point is removed?

I recently took an online quiz on Machine Learning. One question was particularly confusing to me. The question is stated below. Consider a dataset Z on which a decision tree is built. Consider the ...
Bigyan Suryabanshee's user avatar
1 vote
6 answers
264 views

What is the meaning of test data set in naive bayes classifier or decision trees?

What is the benefit of a test data set, especially for naive bayes estimator or decision tree construction? When using a naive bayes classifier the probabilities are a fact. As far as I know there is ...
MScott's user avatar
  • 445
1 vote
1 answer
248 views

At which point we have to stop post pruning in decision tree?

Post pruning is start from downward discarding subtree and include leaf node performance. so what is the best point or condition of the tree where we have to stop further pruning.
hina munir's user avatar
1 vote
1 answer
976 views

What is the intuition behind the entropy formula used in the ID3 algorithm?

What is the intuition behind the following entropy formula used in the ID3 algorithm? $$ \text{info}(D) = -\sum_{i=1}^m p_i \log_2(p_i) $$
DRV's user avatar
  • 1,703
1 vote
1 answer
63 views

What is the concept of pruning a tree in Machine Learning regression problems?

What is the concept of pruning a tree in Machine Learning regression problems? I am confused and a simple explanation would be great.
Shekhar's user avatar
  • 11
1 vote
1 answer
97 views

Why don't we wait if there is no patrons, in this decision tree from Russel and Norvig's book?

I'm reading Russel-Norvig's book about artificial intelligence and now at chapter decision tree where this figure is shown: So far I understood it. This decision tree should answer the question if we ...
Haidepzai's user avatar
  • 141
1 vote
2 answers
996 views

Random forests - are more estimators always better?

I'm learning about more advanced methods of hyperparameter optimization, such as the Bayesian methods in the scikit-optimize package. For those unfamiliar with the ...
SuperCodeBrah's user avatar
1 vote
1 answer
87 views

Can I apply AdaBoost on a random forest?

I know the random forest is a bagging technique. But what if my random forest overfits on a dataset, so I reduce the depth of the decision tree and now it is underfitting. In this scenario, can I take ...
Swakshar Deb's user avatar
1 vote
1 answer
418 views

How to determine if a decision tree is the (globally) optimal tree?

BACKGROUND: When constructing decision trees, the features are selected at various nodes based on whether it optimally splits the samples at that level (i.e., locally) using some user-chosen metric ...
Snehal Patel's user avatar
1 vote
0 answers
22 views

Optimize parametric Log-Likelihood with a Decision Tree

Suppose there are some objects with features, and the target is parametric density estimation. Density estimation is model-based. Parameters are obtained by maximizing log-likelihood. $LL = \sum_{i \...
nekrald's user avatar
  • 11
1 vote
1 answer
477 views

Find the expected reward in an expectimax-based dice rolling game?

I have this question that I'm kinda stuck on. It's a game scenario in which we set up an expectimax tree. In the game, you have 3 dice with sides 1-4 that you roll at the beginning. Then, depending on ...
Manny's user avatar
  • 21
1 vote
2 answers
3k views

What does the depth of a decision tree depend on?

In these notes, we have the following statement The depth of a learned decision tree can be larger than the number of training examples used to create the tree This statement is false, according to ...
S. Christin's user avatar
1 vote
1 answer
99 views

Can the C4.5 algorithm learn a GOAP model?

Goal-oriented action planning (GOAP) is a well-known planning technique in computer games. It was introduced to control the non-player characters in the game F.E.A.R. (2005) by creating an abstract ...
user avatar
0 votes
1 answer
26 views

Is information gain only calculated using decrease in entropy of a dataset?

Is information gain only calculated using decrease in entropy of a dataset? If I calculate it using the decrease in Gini Impurity, before and after the split, can it not be considered information gain?...
Piyush Miglani's user avatar
0 votes
1 answer
274 views

Is there a way to see the feature importance in deep learning (neural networks)? [closed]

For tree methods, I can plot the feature importance plot from tree.feature_importances_ in sklearn, is this achievable in deep learning (neural networks)? Is there ...
user900476's user avatar
0 votes
1 answer
98 views

Does this look like overfitting?

I'm using a Decision Tree that gave me great test metrics. Then I checked the learning curve, but it seems a little strange to me regarding the training score. Do you think there is a problem with ...
giovasbr's user avatar
0 votes
1 answer
536 views

Mathematical calculation behind decision tree classifier with continuous variables

I am working on a binary classification problem having continuous variables (Gene expression Values). My goal is to classify the samples as case or ...
sp29's user avatar
  • 103
0 votes
1 answer
74 views

How many trees should be generated in a random forest?

What are ways of determining the number of trees to be generated in a random forest algorithm?
Iram Shah's user avatar
  • 315
0 votes
1 answer
66 views

Can GenAI be Used to Generate Decision Trees from Text?

I'm exploring the capabilities of GenAI for text analysis and decision-making processes. I'm particularly interested in understanding whether GenAI can be leveraged to create decision trees directly ...
D3N3ON's user avatar
  • 1
0 votes
0 answers
11 views

What model is good for learning both within and across categories?

How can I incorporate both general trends and subcategory-specific trends into a model? Let's say I am predicting factors that affect import volume, for example. There are many industries which have ...
BigMistake's user avatar
0 votes
0 answers
29 views

Confusion about the code for choosing "stumps" in Adaboost algorithm

(I actually asked the following question on Stack Overflow and Cross Validated Exchange for more than a month: https://stackoverflow.com/questions/76842431/confusion-about-the-code-for-choosing-...
Richard's user avatar
  • 101
0 votes
1 answer
39 views

What kind of algorithm to use

For a course term project, we have to build a machine learning algorithm in which the user fills out the form and the algorithm analyses the best suitable university based on the responses. I am new ...
Kanan Suleyman's user avatar
0 votes
1 answer
59 views

How to evaluate binary classifier on imbalanced dataset?

I have trained a Decision Tree model on an imbalanced dataset. I got the following results for the test set from the sklearn and imblearn classification reports (attached below). Moreover, the other ...
Zal's user avatar
  • 7