6
votes
Accepted
What are "development test sets" used for?
In machine learning, you normally split your data into 3 parts (80-10-10%).
The first part (80% of your initial data) is for the training of your ML model: this is known as the training dataset.
The ...
- 274
2
votes
What is the theoretical basis for the use of a validation set?
I think Cross-Validation serves a completely different purpose.
From your post, it looks like you think we would use CV to get a better estimate of the parameters of our model (i.e. the model ...
- 8,877
2
votes
Accepted
What is the difference between validation percentage and batch size?
The percentages refer to the number of samples to use (out of full dataset) as the validation and test datasets. So if you pass a dataset that consists of 100,000 samples to the model and set the ...
- 426
2
votes
Accepted
How to perform PCA in the validation/test set?
for the first point I'm very sorry that I cannot give you any literature on this, but I might be able to explain you, why you don't take PCA on both datasets independently.
Principal components ...
- 96
1
vote
how to decide the optimum model?
Testing each time on a test set is against the point of a train-val-test split. The reason test is important, is that you are only supposed to test on it when you think your model is good and ready ...
- 2,249
1
vote
Why not make the training set and validation set one if their roles are similar?
I think this is best explained using an analogy. Also you seen to have the misconception that you don't tune hyper-parameters for training data. You want to increase the accuracy of the training set ...
- 1,344
1
vote
Why not make the training set and validation set one if their roles are similar?
Idea is to optimize with regards to unseen data in each step in order to avoid overfitting and data leakage so that the final network will be most generalizable to novel data.
First, you initialize ...
- 286
1
vote
Accepted
Are the held-out datasets used for testing, validation or both?
I think that these terms may be used inconsistently across sources.
If someone says held-out dataset, I would immediately think of a dataset that is not used for training, but can be used for anything ...
- 33.8k
Only top scored, non community-wiki answers of a minimum length are eligible