Questions tagged [decision-trees]

For question involving decision trees in any form of AI.

Filter by
Sorted by
Tagged with
0 votes
1 answer
29 views

What kind of algorithm to use

For a course term project, we have to build a machine learning algorithm in which the user fills out the form and the algorithm analyses the best suitable university based on the responses. I am new ...
0 votes
1 answer
54 views

Is there a way to see the feature importance in deep learning (neural networks)? [closed]

For tree methods, I can plot the feature importance plot from tree.feature_importances_ in sklearn, is this achievable in deep learning (neural networks)? Is there ...
0 votes
1 answer
70 views

Does this look like overfitting?

I'm using a Decision Tree that gave me great test metrics. Then I checked the learning curve, but it seems a little strange to me regarding the training score. Do you think there is a problem with ...
2 votes
1 answer
134 views

How to determine if a decision tree is the (globally) optimal tree?

BACKGROUND: When constructing decision trees, the features are selected at various nodes based on whether it optimally splits the samples at that level (i.e., locally) using some user-chosen metric ...
0 votes
0 answers
18 views

Value Of Trees In Decision Tree

I am writing an AI to play a game of "Snake". Whereby two snakes move on a map to eat food to grow, and eventually if a smaller snake runs into a larger snake, it dies. I have built a very ...
0 votes
1 answer
40 views

How to evaluate binary classifier on imbalanced dataset?

I have trained a Decision Tree model on an imbalanced dataset. I got the following results for the test set from the sklearn and imblearn classification reports (attached below). Moreover, the other ...
  • 7
0 votes
1 answer
120 views

How to interpret binary classification metrics on an imbalanced data set?

I have an imbalanced dataset on intrusion detection. I have (attack class) 3668045 samples and (benign class) 477 samples. I made a 70:30 Train test split. My problem is to predict whether the given ...
  • 7
0 votes
0 answers
11 views

Tree boosting additive loss

In the XGBoost documentation, they specify that the additive training is done given an objective $obj^{(t)}$ defined as $obj^{(t)} = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i^{(t-1)}+f_t(x_i)) + \sum\...
0 votes
0 answers
56 views

How to Implement decision tree on FPGA?

I have a large decision tree (depth 80, decision node ~25000 and leaf node ~25000) trained on sklearn decision tree classifier. I am thinking to implement it on an FPGA board. What would be the best ...
  • 19
0 votes
1 answer
63 views

Why don't we wait if there is no patrons, in this decision tree from Russel and Norvig's book?

I'm reading Russel-Norvig's book about artificial intelligence and now at chapter decision tree where this figure is shown: So far I understood it. This decision tree should answer the question if we ...
  • 131
1 vote
1 answer
348 views

Random forests - are more estimators always better?

I'm learning about more advanced methods of hyperparameter optimization, such as the Bayesian methods in the scikit-optimize package. For those unfamiliar with the ...
1 vote
0 answers
18 views

Optimize parametric Log-Likelihood with a Decision Tree

Suppose there are some objects with features, and the target is parametric density estimation. Density estimation is model-based. Parameters are obtained by maximizing log-likelihood. $LL = \sum_{i \...
  • 11
3 votes
1 answer
237 views

Why is the exponential loss used in this case?

I am reading the paper Tracking-by-Segmentation With Online Gradient Boosting Decision Tree. In Section 2.1, the paper says Given training examples, $\left\{\left(\mathbf{x}_{i}, y_{i}\right) \mid \...
2 votes
1 answer
274 views

Is the dropout technique specific only to neural networks?

In one Udemy course was mentioned that "dropout is unique to neural networks". However, I remember an example of decision trees where nodes that are not participating in the overall result ...
2 votes
1 answer
2k views

How does a decision tree split a continuous feature?

Decision trees learn by measuring the quality of a split through some function, apply this to all features and you get the best feature to split on. However, with a continuous feature it becomes ...
  • 1,316
1 vote
1 answer
268 views

Find the expected reward in an expectimax-based dice rolling game?

I have this question that I'm kinda stuck on. It's a game scenario in which we set up an expectimax tree. In the game, you have 3 dice with sides 1-4 that you roll at the beginning. Then, depending on ...
  • 21
1 vote
2 answers
2k views

What does the depth of a decision tree depend on?

In these notes, we have the following statement The depth of a learned decision tree can be larger than the number of training examples used to create the tree This statement is false, according to ...
0 votes
1 answer
325 views

Mathematical calculation behind decision tree classifier with continuous variables

I am working on a binary classification problem having continuous variables (Gene expression Values). My goal is to classify the samples as case or ...
  • 103
2 votes
1 answer
80 views

What are some applications where tree models perform better than neural networks?

Neural networks are known to be generally better modeling techniques as compared to tree-based models (such as decision trees). Are there any exceptions to this?
1 vote
1 answer
66 views

Can I apply AdaBoost on a random forest?

I know the random forest is a bagging technique. But what if my random forest overfits on a dataset, so I reduce the depth of the decision tree and now it is underfitting. In this scenario, can I take ...
2 votes
2 answers
2k views

Why are decision trees and random forests scale invariant?

Feature scaling, in general, is an important stage in the data preprocessing pipeline. Decision Tree and Random Forest algorithms, though, are scale-invariant - i.e. they work fine without feature ...
5 votes
2 answers
557 views

Why isn't my decision tree classifier able to solve the XOR problem properly?

I was trying to solve an XOR problem, and the dataset seems like the one in the image. I plotted the tree and got this result: As I understand, the tree should have depth 2 and four leaves. The ...
2 votes
0 answers
77 views

Why do we use a weighted average of child entropies when we calculate information gain?

In the decision tree algorithm, why do we use a weighted average of child entropies when we calculate information gain? What is wrong about using the arithmetic mean of entropies?
  • 21
1 vote
6 answers
179 views

What is the meaning of test data set in naive bayes classifier or decision trees?

What is the benefit of a test data set, especially for naive bayes estimator or decision tree construction? When using a naive bayes classifier the probabilities are a fact. As far as I know there is ...
  • 445
2 votes
0 answers
586 views

Why information gain with entropy as impurity function can't be used as a splitting method for Decision Tree Regression?

In Decision Tree Regression, we can use 'Reduction in Variance' or MSE (Mean Squared Errors) as splitting methods. There are methods like Gini Index, Information Gain, Chi-Square for splitting on ...
  • 43
2 votes
1 answer
141 views

What is the most suitable AI technique to use for path planning?

I am making a firetruck using Arduino Uno with flame sensors and ultrasonic sensors to detect how to move and where to go. As this is a project for my university, I am asked to implement AI in it for ...
  • 21
3 votes
1 answer
809 views

What is the difference between Inductive Learning and Connectionist Learning?

According to what we know about inductive and connectionist learning, what is the difference between them ? For those who do not know about : Inductive Learning, like what we have in decision tree ...
2 votes
0 answers
86 views

How can I classify instances into two categories and then into sub-categories, when the number of features is high?

I'm working with a problem where I have a lot of variables for different cases of different users. Depending on the values of the different variables of a concrete user in a concrete case, the ...
2 votes
1 answer
57 views

Feature extraction timeseries, model compatibility

I've got a timeseries with sensor data (e.g. accelerometer and gyroscope). I now want to extract the activity out of it (e.g. walking, standing, driving, ...). I Followed this Jupyter Notebook. But ...
2 votes
1 answer
758 views

How does the decision tree implicitly do feature selection?

I was talking with an ex-fellow worker and he told me that the decision tree implicitly applies a feature selection. He told me that the most important feature is higher in the tree because of the ...
5 votes
1 answer
77 views

How could decision tree learning algorithms cope with imbalanced classes?

Decision trees and random forests may or not be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article Using Random Forest to ...
3 votes
1 answer
255 views

“Outside-in” versus “Inside-out” machine learning

A little background... I’ve been on-and-off learning about data science for around a year or so, however, I started thinking about artificial intelligence a few years ago. I have a cursory ...
1 vote
1 answer
85 views

Can the C4.5 algorithm learn a GOAP model?

Goal-oriented action planning (GOAP) is a well-known planning technique in computer games. It was introduced to control the non-player characters in the game F.E.A.R. (2005) by creating an abstract ...
user avatar
3 votes
1 answer
65 views

What are possible functions assigned on decision nodes for decision tree prediction?

In Decision Tree or Random Forest, each tree has a collection of decision nodes (in which each node has a threshold value) and a class labels (or regression values). I know that threshold values are ...
1 vote
1 answer
210 views

At which point we have to stop post pruning in decision tree?

Post pruning is start from downward discarding subtree and include leaf node performance. so what is the best point or condition of the tree where we have to stop further pruning.
10 votes
1 answer
4k views

Are decision tree learning algorithms deterministic?

Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? What about the random forest?
  • 203
2 votes
0 answers
58 views

What kind of decision rule algorithm is usable in this situation?

I am trying to write an AI to a game, where there is no real adversary. This means, that only the AI player has choices in which move to perform, his opponent may or may not react to the move the AI ...
3 votes
2 answers
417 views

What makes a machine learning algorithm a low variance one or a high variance one?

Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression. Examples of high-variance machine learning algorithms ...
  • 338
0 votes
1 answer
67 views

How many trees should be generated in a random forest?

What are ways of determining the number of trees to be generated in a random forest algorithm?
  • 305
2 votes
2 answers
114 views

How can I minimize the number of answers that are relevant to a machine learning model?

Problem: We have a fairly big database that is built up by our own users. The way this data is entered is by asking the users 30ish questions that all have around 12 answers (x, a, A, B, C, ..., H). ...
1 vote
1 answer
500 views

Why was Go a harder game for an AI to master than Chess?

AI became superior to the best human players in chess around 20 years ago (when the 2nd Deep Blue match concluded). However, it took until 2016 for an AI to beat the Go world chess champion, and this ...
4 votes
1 answer
165 views

Decision tree: more than 2 classes, how to represent elements that are in a class vs ones that aren't?

I'm building a decision tree and would like to separate (for example) the elements that are in class 0 from those in classes 1 and 2, case in point: ...
2 votes
1 answer
531 views

What do the values of the leaves of the decision tree represent?

This is more of a technical question rather than a practical one. I've exported a decision tree made with python/scikit learn and would like to know what the "value" field of each leaf corresponds to....
9 votes
1 answer
393 views

Why does nobody use decision trees for visual question answering?

I'm starting a project that will involve computer vision, visual question answering, and explainability. I am currently choosing what type of algorithm to use for my classifier - a neural network or a ...
4 votes
1 answer
2k views

Given a dataset with no noisy examples, is the training error for the ID3 algorithm always 0?

Given a dataset with no noisy examples (i.e., it is never the case that for 2 examples, the attribute values match but the class value does not), is the training error for the ID3 algorithm is always ...
  • 105
1 vote
1 answer
910 views

What is the intuition behind the entropy formula used in the ID3 algorithm?

What is the intuition behind the following entropy formula used in the ID3 algorithm? $$ \text{info}(D) = -\sum_{i=1}^m p_i \log_2(p_i) $$
  • 1,433
2 votes
3 answers
164 views

Why are tree-based models more widely used in Medical Diagnosis?

In Chapter 14.4 (p. 664) of the book Pattern Recognition and Machine Learning by Bishop, it is mentioned that tree-based models are more widely used in Medical Diagnosis. Apart from giving better ...
3 votes
2 answers
5k views

How to calculate the entropy in the ID3 decision tree algorithm?

Here is the definition of the entropy $$H(S)=-\sum_{x \in X} p(x) \log _{2} p(x)$$ Wikipedia's description of entropy breaks down the formula, but I still don't know how to determine the values of $X$,...