In almost every ML model, a train-test (or train-test-val split) is critical to assess the model's performance. However, I have always wondered what the rationale is to decide a particular train-test split. I've seen that some people like an 80-20 split, others opt for 90-10, but why? Is it simply a matter of preference? Also, why not 70-30 or 60-40, what is the best way to decide?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
I don't think there is any rationale behind choosing 80/20 over 75/25 or others. But those are the numbers for rather small datasets. If your dataset is large enough (like hundreds of thousands of samples), you can even work with 98/1/1 percents for train/val/test as discussed by Andrew Ng in this video. Neural networks thrive with big data and it is always a good idea to make use most of it.
