2
$\begingroup$

On this website https://scikit-learn.org/stable/modules/learning_curve.html, the authors are speaking about variance and bias and they give a simple example of how works in a linear model.

How can I determine the bias and variance of a random forest?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods.

A great article on this topic that I recommend as a light mathematical introduction is this: https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229

In short: Bias represents the models effort to generalize samples, as opposed to Variance that represents the models effort to conform to new data. A high bias, low variance model will thus look more like a straight(underfitted) line, while a low bias, high variance model will look jagged and all-over the place(overfitted).

In essence, you need to find a balance between the two to avoid both overfitting(high variance, low bias) and underfitting(high bias, low variance) for your specific application.

But how can I determine this for a model such as a Random Forrest classifier?

To determine your models bias and variance configuration(if either is too high/low), you can look at the models performance on the validation and test set. The very reason we divide our data into training-validation-test sets, is so that we can validate the models performance when it is presented with samples it has not seen during training.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Krrrl I am familiar with the famous dartboard and for basic linear models I do understand how it works, . And the test method that you suggest I am familar with but ai.stackexchange.com/questions/15964/… here look in the comments the opposite is said (it can be underfitting and overfitting so i am confused) $\endgroup$
    – jennifer ruurs
    Commented Oct 22, 2019 at 14:52
  • 1
    $\begingroup$ Sorry, I did not mean to imply that you did not already know that. I am not exactly sure what your question is in your comment, could you type it as a question? The graphs you posted in the other question seem to indicate that the model is overfitting with a tree-depth higher than 15. That is to say, the useful features in the data is already accounted for by depth 15, thus by increasing depth above 15 the model starts to learn features unimportant to the actual data(such as noise present in your training set). Does this answer your question? $\endgroup$
    – Krrrl
    Commented Oct 22, 2019 at 15:02
  • 1
    $\begingroup$ No, that is not the case. The graph you posted in the other question illustrates what accuracy is achieved for different tree depths - tested on two different data sets(training set and validation set). You should use this graph to determine the depth of the trees in your model, before you are going to use it on real data(non-training data). So, the graph tells you that a depth of 15 seem to be the optimal depth for the specific data you are working with here. If you change out the data set with something else, you will probably get a new optimal tree depth, specific to the new data. $\endgroup$
    – Krrrl
    Commented Oct 22, 2019 at 15:37
  • 1
    $\begingroup$ Think of the graph like this: When different tree depths are tested, these different accuracy's are found. If you then choose tree depth deeper than 15(say 18), your model will be overfitting. While if you choose a tree depth less than 15(say 12), you model will be underfitting. So, you use this graph to determine how deep your trees should be - in order to avoid both under- and overfitting. The two graphs are the result of using the same model, tested with different tree depths, on two different data sets(the training- and the validation-set). $\endgroup$
    – Krrrl
    Commented Oct 22, 2019 at 15:43
  • 1
    $\begingroup$ Thank you for editing your previous comment for clarity. I am not aware of any way to compute over/underfitting analytically, other than comparing performance on trainingset vs. validation set(which is what we were talking about earlier, in your original question). The learning curve can indeed be used to conclude something about over/underfitting, based on the difference between performance on training and validation set, like demonstrated in the article you linked. $\endgroup$
    – Krrrl
    Commented Oct 22, 2019 at 16:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .