How to avoid over-fitting using early stopping when using R cross validation package caret

Question

I have a data set with 36 rows and 9 columns. I am trying to make a model to predict the 9th column

I have tried modeling the data using a range of models using caret to perform cross-validation and hyper parameter tuning: 'lm', random forrest (ranger) and GLMnet, with range of different folds and hyper-parameter tuning, but the modeling has not been very successful.

Next I have tried to use some of the neural-network models. I tried the 'monmlp'. During hyper parameter tuning I could see that the RMSE drops to a level when using ~ 6 hidden units. The problem I observe using this model is

1. Prediction is almost equal to data
2. When doing a "manual" cross validation by removing a single datapoint and using the trained model to predict, it has no predictive power

I have tried to use a range of different hidden units, but i think the problem is that the model is overfitted despite using caret cross validation feature.

There two feedbacks I would appreciate

1. Is there a way to prevent overfitting, by chosen optimal number of training iterations ( optimal RMSE on out of sample ). Can this by done using caret or some other package
2. Am I using the right model?

I am relatively unexperienced with ML and choosing a good model is tough: when you look at the available packages it is overwhelming:

https://topepo.github.io/caret/train-models-by-tag.html

Answer 1

You are handling a very small dataset. The only way to prevent overfitting then is to choose a very restrictive model search space. The simpler the better, and you should prefer models involving some regularization. Even tuning hyperparameters will be hard, so avoid families with many hyperparameters. Neural networks are definitely a no-go, IMHO.

Cross-validation is important in this case, a.o. for tuning the regularization hyperparameter. You were right to perform the extra leave-one-out cross-validation. But remember checking the confidence intervals, which will be very broad.

I would suggest to use some prior business knowledge to further reduce the number of candidate predictor variables to 2-4. Then you can use a linear regression or a ridge regression (i.e. linear regression with L2 regularization, amount to be tuned as hyperparameter) using only these variables. If you have reasons to believe that the relation is very non-linear, you'll need to find another family.

Another possibility is to perform a data-driven feature selection (see https://topepo.github.io/caret/feature-selection-overview.html), but it is harder to properly implement and cross-validate. I wouldn't recommend it to a beginner. Remember that this step can also introduce overfit or instability.

Answer 2

Thanks for your answer. I have tried to assess other models using cross validation and could see that the complex neural network models did not perform very well on out of sample data.. what tricked me was that i looked at the predicted data from the training on the full data set ( after optimization of hyper parameter tuning). This predicted data alwasys looked extremely well correlated with actual data. I did not look at the magnitude of the RMSE after hyperparamter tuning.Because. When I then did an extra "leave-one-out" validation i could get random results.

In the end I used caret's resampling method which compares RMSE of different models.and found out that svmPoly (support vector machine) or random forrest had the best out of sample performance - better than lm and GLMnet.