Questions tagged [validation-datasets]

For questions related to validation datasets, which are used for hyper-parameter optimization, early stopping or cross-validation. They are sometimes referred to as held-out datasets, but this latter term could also be used to refer to test datasets (i.e. the datasets used to assess the generalisation of the model).

Filter by
Sorted by
Tagged with
0 votes
0 answers
28 views

I dont understand this way of having a stable train/test split even after updating the dataset

...
samsamradas's user avatar
0 votes
2 answers
81 views

Is it legitimate to train a model on a benchmark dataset and use this model only for labeling another datasets

To assess our deep learning models (CNN) we have labeled a big benchmark dataset (it was labeled by specialists so it is kind of Ideal). I of course know that we do not want to train new models using ...
Igor's user avatar
  • 181
1 vote
1 answer
31 views

Is there validation data in K-fold cross-validation?

We know that in machine learning the dataset is divided into 3 parts: training data, validation data and test data. On the other hand, K-fold cross-validation is defined as follows: the dataset is ...
DSPinfinity's user avatar
3 votes
4 answers
1k views

How is MNIST only providing the training and the test sets? What about the validation?

I was taught that, usually, a dataset has to be divided into three parts: Training set - for learning purposes Validation set - for picking the model which minimize the loss on this set Test test - ...
tail's user avatar
  • 157
1 vote
1 answer
96 views

Datasets input at model.fit produce unexpected results of training loss vs validation loss

Im trying to train a neural network (VAE) using tensorflow and Im getting different results based on the type of input in the model.fit. When I input arrays I get normal difference between the ...
user avatar
-1 votes
1 answer
66 views

how to decide the optimum model?

I have split the database available into 70% training, 15% validation, and 15% test, using holdout validation. I have trained the model and got the following results: training accuracy 100%, ...
user50778's user avatar
2 votes
2 answers
74 views

Why not make the training set and validation set one if their roles are similar?

If the validation set is used to tune the hyperparameters and the training set adjusts the weights, why don't they be one thing as they have a similar role, as in improving the model?
Omar Zayed's user avatar
1 vote
2 answers
350 views

Are the held-out datasets used for testing, validation or both?

I came across a new term "held-out corpora" and I confused regarding its usage in the NLP domain Consider the following three paragraphs from N-gram Language Models #1: held-out corpora as a ...
hanugm's user avatar
  • 3,840
1 vote
1 answer
66 views

What is the theoretical basis for the use of a validation set?

Let's say we use an MLE estimator (implementation doesn't matter) and we have a training set. We assume that we have sampled the training set from a Gaussian distribution $\mathcal N(\mu, \sigma^2)$. ...
user avatar
2 votes
1 answer
468 views

What is the difference between validation percentage and batch size?

I'm doing transfer learning using Inception on Tensorflow. The code that I used for training is https://raw.githubusercontent.com/tensorflow/hub/master/examples/image_retraining/retrain.py If you take ...
gameon67's user avatar
  • 215
1 vote
1 answer
984 views

How to perform PCA in the validation/test set?

I was using PCA on my whole dataset (and, after that, I would split it into training, validation, and test datasets). However, after a little bit of research, I found out that this is the wrong way to ...
LVoltz's user avatar
  • 131
4 votes
1 answer
221 views

What are "development test sets" used for?

This is a theoretical question. I am a newbie to artificial intelligence and machine learning, and the more I read the more I like this. So far, I have been reading about the evaluation of language ...
little_mice's user avatar