1
$\begingroup$

Bellow I have a validation plot How should I interpret this validation plot? Is my data underfitting? What else can be seen from this? Which one is the best?

What does it mean that the right line is growing and green line decrease (slightly) for example after 15?

enter image description here

Second random forrest enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

This is a sign of overfitting.

As you make your trees deeper, it becomes possible to "memorize" the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not exist. When you try out these patterns on new data (which is what cross-validation is imitating), then the patterns do not work, and your model fails to generalize.

The main piece of information to draw from this plot is that the optimum tree depth is about 15.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Isn't the opposite true, that I am underfitting? Because the we do first cross-validation and evaluate on the test set how the cross-validation model behaves on the test set. If the scores of the test set is lower than cross-validation than it is overfitting? $\endgroup$
    – jennifer ruurs
    Commented Oct 19, 2019 at 5:53
  • $\begingroup$ And is the second one better than the first one? $\endgroup$
    – jennifer ruurs
    Commented Oct 19, 2019 at 11:53
  • $\begingroup$ @jenniferruurs Lower scores on the test set than the training set can mean overfitting or underfitting. Your plot tells us that depth values below 15 are probably underfitting: we can improve performance by increasing model complexity. Depth values above 15 are probably overfitting: we can improve performance by reducing model complexity. Neither line is "better" than the other. They mean different things. $\endgroup$
    – John Doucette
    Commented Oct 19, 2019 at 12:45
  • $\begingroup$ I mean i added a second plot, and is the second plot a better model than the first one? $\endgroup$
    – jennifer ruurs
    Commented Oct 19, 2019 at 12:54
  • $\begingroup$ In assessing model quality, you should only use test it validation performance. In both of your graphs, the cross validation curves are essentially the same. The best models occur at a depth of 15, and they have about 84% accuracy in both cases. $\endgroup$
    – John Doucette
    Commented Oct 19, 2019 at 18:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .