Sorry if I sound confused. I read that data to be fed to a machine are divided into training, validation and test data. Both training and validation data are used for developing the model. Test data is used only for testing the model and no tuning of the model is done using test data.
Why is there a need to separate out training and validation data since both sets of data are for developing/tuning the model? Why not keep things simple and combine both data sets into a single one?
Usually we are splinting the training dataset because we want to fine tune, find the best hyper-parameters, for our model. If we combine the validation and training dataset given a network complex enough we could achieve perfect performance for the given task. But having very good performance on the training dataset does not mean our model is useful. It might be the case that our model is very good on the training dataset, but it fails to generalize, so when given samples outside of the training dataset the performance will significantly drop (this is called over-fitting). By splitting the dataset, we are testing how well our model is performing on new/unseen data.
There are two possible forms of overfitting. First related to the training only (fitting weights) and second related to architecture (fitting hyperparameters) and these two must be checked in two different stages. When you check performance of the given model you have two do this on unseen data, so you fit weights on training data and check it on validation (unseen to the weights fitting process) set and next you fit hyperparameters on validation set and check it on test (unseen to the hyperparameters fitting process) set.
