I'm currently working on a project in material science and the data to evaluate is very limited. I work with about 60 datasets, each with about 10.000 relevant lines.
However I want to predict a certain parameter which is not always measured nor frequently. To be exact: I have about 30 lines for that parameter spread across 8 of the 60 datasets.
I'm currently thinking of a workaround: I've seen solutions like simulating more data, but this is not possible in my case.
My idea was to set only these 8 datasets as validation data (or cross validate with 2/4 sets at a time). The rest would be always used for training.
Is this a viable approach? I'd also train more than one model with different approaches and then evaluate them all together. But I'm worried if the model will only predict the validation set as correct and the other ones are wrong, without me ever knowing. Since it's a costly process, I'll probably only get a few more datasets in the future.
Thanks in advance!
