Questions tagged [decision-trees]
For question involving decision trees in any form of AI.
53
questions
0
votes
1
answer
53
views
What is the best machine learning textbook for decision trees?
What is the best machine learning textbook for decision trees, where theory and
practice are balanced?
- 973
0
votes
0
answers
19
views
How can I train a decision tree or simple ML model with spike detections
I am looking for some advice for my problem. I have a spike detection algorithm that detections outliers in a signal. I have annotations in my dataset that tells me if I am in 3 events, basal, event A ...
- 1
0
votes
2
answers
124
views
Can GenAI be Used to Generate Decision Trees from Text?
I'm exploring the capabilities of GenAI for text analysis and decision-making processes. I'm particularly interested in understanding whether GenAI can be leveraged to create decision trees directly ...
- 1
0
votes
0
answers
12
views
What model is good for learning both within and across categories?
How can I incorporate both general trends and subcategory-specific trends into a model?
Let's say I am predicting factors that affect import volume, for example. There are many industries which have ...
- 113
0
votes
1
answer
29
views
Is information gain only calculated using decrease in entropy of a dataset?
Is information gain only calculated using decrease in entropy of a dataset?
If I calculate it using the decrease in Gini Impurity, before and after the split, can it not be considered information gain?...
1
vote
1
answer
68
views
What is the concept of pruning a tree in Machine Learning regression problems?
What is the concept of pruning a tree in Machine Learning regression problems? I am confused and a simple explanation would be great.
- 11
1
vote
1
answer
235
views
Does the root node of a Decision Tree change when one data point is removed?
I recently took an online quiz on Machine Learning. One question was particularly confusing to me. The question is stated below.
Consider a dataset Z on which a decision tree is built. Consider the ...
0
votes
0
answers
32
views
Confusion about the code for choosing "stumps" in Adaboost algorithm
(I actually asked the following question on Stack Overflow and Cross Validated Exchange for more than a month:
https://stackoverflow.com/questions/76842431/confusion-about-the-code-for-choosing-...
- 101
0
votes
1
answer
39
views
What kind of algorithm to use
For a course term project, we have to build a machine learning algorithm in which the user fills out the form and the algorithm analyses the best suitable university based on the responses. I am new ...
0
votes
1
answer
305
views
Is there a way to see the feature importance in deep learning (neural networks)? [closed]
For tree methods, I can plot the feature importance plot from tree.feature_importances_ in sklearn, is this achievable in deep learning (neural networks)?
Is there ...
- 121
0
votes
1
answer
99
views
Does this look like overfitting?
I'm using a Decision Tree that gave me great test metrics. Then I checked the learning curve, but it seems a little strange to me regarding the training score. Do you think there is a problem with ...
- 21
1
vote
1
answer
449
views
How to determine if a decision tree is the (globally) optimal tree?
BACKGROUND: When constructing decision trees, the features are selected at various nodes based on whether it optimally splits the samples at that level (i.e., locally) using some user-chosen metric ...
- 987
0
votes
1
answer
62
views
How to evaluate binary classifier on imbalanced dataset?
I have trained a Decision Tree model on an imbalanced dataset. I got the following results for the test set from the sklearn and imblearn classification reports (attached below). Moreover, the other ...
- 7
0
votes
1
answer
152
views
How to interpret binary classification metrics on an imbalanced data set?
I have an imbalanced dataset on intrusion detection. I have (attack class) 3668045 samples and (benign class) 477 samples. I made a 70:30 Train test split. My problem is to predict whether the given ...
- 7
1
vote
1
answer
99
views
Why don't we wait if there is no patrons, in this decision tree from Russel and Norvig's book?
I'm reading Russel-Norvig's book about artificial intelligence and now at chapter decision tree where this figure is shown:
So far I understood it. This decision tree should answer the question if we ...
- 141
1
vote
2
answers
1k
views
Random forests - are more estimators always better?
I'm learning about more advanced methods of hyperparameter optimization, such as the Bayesian methods in the scikit-optimize package. For those unfamiliar with the ...
- 273
1
vote
0
answers
22
views
Optimize parametric Log-Likelihood with a Decision Tree
Suppose there are some objects with features, and the target is parametric density estimation. Density estimation is model-based. Parameters are obtained by maximizing log-likelihood.
$LL = \sum_{i \...
- 11
3
votes
1
answer
458
views
Why is the exponential loss used in this case?
I am reading the paper Tracking-by-Segmentation With Online Gradient Boosting Decision Tree. In Section 2.1, the paper says
Given training examples, $\left\{\left(\mathbf{x}_{i}, y_{i}\right) \mid \...
- 33
2
votes
1
answer
299
views
Is the dropout technique specific only to neural networks?
In one Udemy course was mentioned that "dropout is unique to neural networks". However, I remember an example of decision trees where nodes that are not participating in the overall result ...
- 125
2
votes
1
answer
3k
views
How does a decision tree split a continuous feature?
Decision trees learn by measuring the quality of a split through some function, apply this to all features and you get the best feature to split on.
However, with a continuous feature it becomes ...
- 1,406
1
vote
1
answer
487
views
Find the expected reward in an expectimax-based dice rolling game?
I have this question that I'm kinda stuck on.
It's a game scenario in which we set up an expectimax tree. In the game, you have 3 dice with sides 1-4 that you roll at the beginning. Then, depending on ...
- 21
1
vote
2
answers
3k
views
What does the depth of a decision tree depend on?
In these notes, we have the following statement
The depth of a learned decision tree can be larger than the number of training examples used to create the tree
This statement is false, according to ...
- 13
0
votes
1
answer
551
views
Mathematical calculation behind decision tree classifier with continuous variables
I am working on a binary classification problem having continuous variables (Gene expression Values). My goal is to classify the samples as case or ...
- 103
2
votes
1
answer
88
views
What are some applications where tree models perform better than neural networks?
Neural networks are known to be generally better modeling techniques as compared to tree-based models (such as decision trees). Are there any exceptions to this?
- 39
1
vote
1
answer
88
views
Can I apply AdaBoost on a random forest?
I know the random forest is a bagging technique. But what if my random forest overfits on a dataset, so I reduce the depth of the decision tree and now it is underfitting. In this scenario, can I take ...
- 693
2
votes
2
answers
3k
views
Why are decision trees and random forests scale invariant?
Feature scaling, in general, is an important stage in the data preprocessing pipeline.
Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature ...
- 1,171
5
votes
2
answers
959
views
Why isn't my decision tree classifier able to solve the XOR problem properly?
I was trying to solve an XOR problem, and the dataset seems like the one in the image.
I plotted the tree and got this result:
As I understand, the tree should have depth 2 and four leaves. The ...
- 53
3
votes
0
answers
103
views
Why do we use a weighted average of child entropies when we calculate information gain?
In the decision tree algorithm, why do we use a weighted average of child entropies when we calculate information gain? What is wrong about using the arithmetic mean of entropies?
- 31
1
vote
6
answers
276
views
What is the meaning of test data set in naive bayes classifier or decision trees?
What is the benefit of a test data set, especially for naive bayes estimator or decision tree construction?
When using a naive bayes classifier the probabilities are a fact. As far as I know there is ...
- 445
2
votes
0
answers
753
views
Why information gain with entropy as impurity function can't be used as a splitting method for Decision Tree Regression?
In Decision Tree Regression, we can use 'Reduction in Variance' or MSE (Mean Squared Errors) as splitting methods. There are methods like Gini Index, Information Gain, Chi-Square for splitting on ...
- 43
2
votes
1
answer
164
views
What is the most suitable AI technique to use for path planning?
I am making a firetruck using Arduino Uno with flame sensors and ultrasonic sensors to detect how to move and where to go. As this is a project for my university, I am asked to implement AI in it for ...
- 21
3
votes
1
answer
857
views
What is the difference between Inductive Learning and Connectionist Learning?
According to what we know about inductive and connectionist learning, what is the difference between them ?
For those who do not know about :
Inductive Learning, like what we have in decision tree ...
- 209
2
votes
0
answers
92
views
How can I classify instances into two categories and then into sub-categories, when the number of features is high?
I'm working with a problem where I have a lot of variables for different cases of different users. Depending on the values of the different variables of a concrete user in a concrete case, the ...
- 121
2
votes
1
answer
64
views
Feature extraction timeseries, model compatibility
I've got a timeseries with sensor data (e.g. accelerometer and gyroscope). I now want to extract the activity out of it (e.g. walking, standing, driving, ...). I Followed this Jupyter Notebook. But ...
2
votes
1
answer
1k
views
How does the decision tree implicitly do feature selection?
I was talking with an ex-fellow worker and he told me that the decision tree implicitly applies a feature selection. He told me that the most important feature is higher in the tree because of the ...
- 589
5
votes
1
answer
88
views
How could decision tree learning algorithms cope with imbalanced classes?
Decision trees and random forests may or not be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article Using Random Forest to ...
- 589
3
votes
1
answer
327
views
“Outside-in” versus “Inside-out” machine learning
A little background... I’ve been on-and-off learning about data science for around a year or so, however, I started thinking about artificial intelligence a few years ago. I have a cursory ...
- 273
1
vote
1
answer
100
views
Can the C4.5 algorithm learn a GOAP model?
Goal-oriented action planning (GOAP) is a well-known planning technique in computer games. It was introduced to control the non-player characters in the game F.E.A.R. (2005) by creating an abstract ...
user11571
3
votes
1
answer
86
views
What are possible functions assigned on decision nodes for decision tree prediction?
In Decision Tree or Random Forest, each tree has a collection of decision nodes (in which each node has a threshold value) and a class labels (or regression values).
I know that threshold values are ...
- 133
1
vote
1
answer
249
views
At which point we have to stop post pruning in decision tree?
Post pruning is start from downward discarding subtree and include leaf node performance. so what is the best point or condition of the tree where we have to stop further pruning.
- 187
10
votes
1
answer
5k
views
Are decision tree learning algorithms deterministic?
Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure?
What about the random forest?
- 203
2
votes
0
answers
63
views
What kind of decision rule algorithm is usable in this situation?
I am trying to write an AI to a game, where there is no real adversary. This means, that only the AI player has choices in which move to perform, his opponent may or may not react to the move the AI ...
- 143
3
votes
2
answers
502
views
What makes a machine learning algorithm a low variance one or a high variance one?
Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression.
Examples of high-variance machine learning algorithms ...
- 368
0
votes
1
answer
74
views
How many trees should be generated in a random forest?
What are ways of determining the number of trees to be generated in a random forest algorithm?
- 315
3
votes
2
answers
187
views
How can I minimize the number of answers that are relevant to a machine learning model?
Problem:
We have a fairly big database that is built up by our own users.
The way this data is entered is by asking the users 30ish questions that all have around 12 answers (x, a, A, B, C, ..., H). ...
1
vote
1
answer
720
views
Why was Go a harder game for an AI to master than Chess?
AI became superior to the best human players in chess around 20 years ago (when the 2nd Deep Blue match concluded). However, it took until 2016 for an AI to beat the Go world chess champion, and this ...
4
votes
1
answer
203
views
Decision tree: more than 2 classes, how to represent elements that are in a class vs ones that aren't?
I'm building a decision tree and would like to separate (for example) the elements that are in class 0 from those in classes 1 and 2, case in point:
...
- 253
2
votes
1
answer
847
views
What do the values of the leaves of the decision tree represent?
This is more of a technical question rather than a practical one.
I've exported a decision tree made with python/scikit learn and would like to know what the "value" field of each leaf corresponds to....
- 253
9
votes
1
answer
488
views
Why does nobody use decision trees for visual question answering?
I'm starting a project that will involve computer vision, visual question answering, and explainability. I am currently choosing what type of algorithm to use for my classifier - a neural network or a ...
4
votes
1
answer
3k
views
Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?
Given a dataset with no noisy examples (i.e., it is never the case that for 2 examples, the attribute values match but the class value does not), is the training error for the ID3 algorithm is always ...
- 105