I have a data set with 36 rows and 9 columns. I am trying to make a model to predict the 9th column
I have tried modeling the data using a range of models using caret to perform cross-validation and hyper parameter tuning: 'lm', random forrest (ranger) and GLMnet, with range of different folds and hyper-parameter tuning, but the modeling has not been very successful.
Next I have tried to use some of the neural-network models. I tried the 'monmlp'. During hyper parameter tuning I could see that the RMSE drops to a level when using ~ 6 hidden units. The problem I observe using this model is
- Prediction is almost equal to data
- When doing a "manual" cross validation by removing a single datapoint and using the trained model to predict, it has no predictive power
I have tried to use a range of different hidden units, but i think the problem is that the model is overfitted despite using caret cross validation feature.
There two feedbacks I would appreciate
- Is there a way to prevent overfitting, by chosen optimal number of training iterations ( optimal RMSE on out of sample ). Can this by done using caret or some other package
- Am I using the right model?
I am relatively unexperienced with ML and choosing a good model is tough: when you look at the available packages it is overwhelming: