Tag Info

In machine learning, you normally split your data into 3 parts (80-10-10%). The first part (80% of your initial data) is for the training of your ML model: this is known as the training dataset. The second part (10%) is the development set (or dataset), aka validation set. This is used as measuring your performance with various hyperparameters (e.g. in ...

for the first point I'm very sorry that I cannot give you any literature on this, but I might be able to explain you, why you don't take PCA on both datasets independently. Principal components analysis is simply a transformation of your data into another (less dimensional) coordinate system. The axis for your new coordinate system are defined by the ...

I think this is best explained using an analogy. Also you seen to have the misconception that you don't tune hyper-parameters for training data. You want to increase the accuracy of the training set AND validation set at the same time, but the validation set is more important so you want to maximise that accuracy more. Imagine you had a toddler, and you were ...

I think that these terms may be used inconsistently across sources. If someone says held-out dataset, I would immediately think of a dataset that is not used for training, but can be used for anything else, validation (hyper-parameter tuning or early stopping) or testing; so, to determine what they are referring to, I would probably take into account the ...