5

Are decision tree learning algorithms deterministic? Given a fixed dataset, do they always produce a tree with the same structure? Generally, yes. Most decision tree learners, like the common ID3 and C4.5/C5.0 algorithms, are deterministic. At each step, the learners consider all possible features that have not yet been used to split the data, and find the ...


3

Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem. If you look carefully at the three sources you post, you will find that they actually all agree on this point. Two of the sources actually propose methods of addressing this ...


2

Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector machines, neural networks, distance based clustering methods (e.g. k means) and linear/logistic regression are prone to changes by feature scaling. Those which are ...


2

Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number. Imagine: For a feature that goes from 0 to 100 a cut at 50 may be improving performance. Scaling this down to 0 to 1 making the cut a 0.5 doesn't change a thing. Now on the other hand NN have some kind of activation function ...


2

There are two main things to consider for dealing with imbalanced data: During Training: Undersampling the majority class (healthy patients) so that the model is not that biased to predicting healthy During Evaluation: Using a suitable metric to try to evaluate your model and try to optimize on when you are fine-tuning your random-forest. For imbalanced ...


2

A random forest is a collection of classification trees. If more than 50% of these trees predict class A (and not class B), the random forest will predict class A. What you can do is lower the percentage needed to classify it as class A (in your case, patient has the virus). This way, you can tell your random forest to predict class A if only 20% (or 10%, ...


2

This is a sign of overfitting. As you make your trees deeper, it becomes possible to "memorize" the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not exist. When you try out these patterns on new data (which is what cross-validation is imitating), then the patterns do not work, and your model fails to ...


2

One possible reason may have something to do with the scrutability of models, as described in the first few paragraphs of this article. It presents a case study of a hospital whose policy was to send asthma sufferers to an intensive care unit; the intensive care meant they were less likely to develop pneumonia and therefore the data showed that people with ...


2

Short Intro It's very common for people to think that Deep Learning is a "superior form" of Neural Network, a "smarter model". And then they try to use DL for solving simple tasks and they'll find more problems than solutions. We might think of plane as superior to a bike. But when we need to buy some bread for breakfast, taking a plane ...


1

For the ANN, it should be the average of the error per instance from testing (prediction) when each instance is left out of training. ANNs can unfortunately learn based on the order of instances used for training, so it helps to train/test and then shuffle (permute, or randomly re-order) and then assign to k-folds, then train/test again in order to prevent ...


1

Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see here and here), I would say tree-based models e.g. Random Forests, Decision Trees, Gradient Boosting are favorable over neural networks when working with low-dimensional data and easy interpretable features (usually simple tabular data with ...


1

To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods. A great article on this topic that I recommend as a light mathematical introduction is this: https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229 In short: Bias represents the models effort to ...


1

Note the X index is training set size. For the first and second case, teh training set size starts at 0(or 1). The model will overfit certainly at that data size. When data size increases, the model overfits less and less and eventually the model have enough data samples that it won't overfit. The data size continue to increase and the model performance ...


1

The number of estimators in Random Forest is a hyper-parameter. If you are using SKLearn's Random Classifier you can use one of the following techniques to find a (near) optimal hyperparameter settings (Note:You can tweak other hyperparameters like min_leaf_size etc as well with this approach); GridSearchCV You can specify a grid of all the hyperparameters ...


Only top voted, non community-wiki answers of a minimum length are eligible