1
$\begingroup$

Bellow I have a validation plot How should I interpret this validation plot? Is my data underfitting? What else can be seen from this? Which one is the best?

What does it mean that the right line is growing and green line decrease (slightly) for example after 15?

enter image description here

Second random forrest enter image description here

$\endgroup$
2
$\begingroup$

This is a sign of overfitting.

As you make your trees deeper, it becomes possible to "memorize" the data: each leaf of the tree is just a single point. The trees begin to learn patterns that do not exist. When you try out these patterns on new data (which is what cross-validation is imitating), then the patterns do not work, and your model fails to generalize.

The main piece of information to draw from this plot is that the optimum tree depth is about 15.

$\endgroup$
  • $\begingroup$ Isn't the opposite true, that I am underfitting? Because the we do first cross-validation and evaluate on the test set how the cross-validation model behaves on the test set. If the scores of the test set is lower than cross-validation than it is overfitting? $\endgroup$ – jennifer ruurs Oct 19 '19 at 5:53
  • $\begingroup$ And is the second one better than the first one? $\endgroup$ – jennifer ruurs Oct 19 '19 at 11:53
  • $\begingroup$ @jenniferruurs Lower scores on the test set than the training set can mean overfitting or underfitting. Your plot tells us that depth values below 15 are probably underfitting: we can improve performance by increasing model complexity. Depth values above 15 are probably overfitting: we can improve performance by reducing model complexity. Neither line is "better" than the other. They mean different things. $\endgroup$ – John Doucette Oct 19 '19 at 12:45
  • $\begingroup$ I mean i added a second plot, and is the second plot a better model than the first one? $\endgroup$ – jennifer ruurs Oct 19 '19 at 12:54
  • $\begingroup$ In assessing model quality, you should only use test it validation performance. In both of your graphs, the cross validation curves are essentially the same. The best models occur at a depth of 15, and they have about 84% accuracy in both cases. $\endgroup$ – John Doucette Oct 19 '19 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.