Questions tagged [test-datasets]
For questions related to test (or testing) datasets in the context of machine learning. A test dataset is any dataset that is not used for training the model but just to evaluate it, in particular, its ability to generalize to unseen data.
16
questions
0
votes
0
answers
26
views
How can validation accuracy be more than test accuracy?
I have been trying to implement DenseNet on small dataset using k-fold cross validation. Training accuracy is 94% ,validation accuracy is 73% whereas test accuracy is 90%.I have taken 10% of my total ...
- 13
1
vote
0
answers
69
views
Why does the SVM perform poorly on test data that has a different class distribution than the training data?
Do you know why the SVM performs poorly on test data that has a different class distribution than the training data? The training data has around 15 classes, and the additional testing data has around ...
- 11
0
votes
2
answers
123
views
Why is the WMT16 dataset favoured for evaluating machine translation models?
The Workshop on Statistical Machine Translation has released translation challenges each year from 2004 on, which feature a dataset of sentence pairs in a variety of languages.
Even though the ...
- 101
0
votes
1
answer
66
views
Given a dataset of people with and without cancer, should I split it into training and test datasets such that the same person is not in both?
I have a database that contains healthy persons and lung cancer patients. I need to design a deep neural network for the binary classification problem (cancer/no cancer). I need to split the dataset ...
- 111
-1
votes
1
answer
65
views
how to decide the optimum model?
I have split the database available into 70% training, 15% validation, and 15% test, using holdout validation. I have trained the model and got the following results: training accuracy 100%, ...
1
vote
1
answer
758
views
What does it mean by overfitting the test set?
Consider the following statement from p14 of Naive Bayes and Sentiment Classification
While the use of a devset avoids overfitting the test set, having a
fixed training set, devset, and test set ...
- 3,581
1
vote
2
answers
270
views
Are the held-out datasets used for testing, validation or both?
I came across a new term "held-out corpora" and I confused regarding its usage in the NLP domain
Consider the following three paragraphs from N-gram Language Models
#1: held-out corpora as a ...
- 3,581
0
votes
1
answer
88
views
What are possible ways to combat overfitting or improve the test accuracy in my case?
I have asked a question here, and one of the comments suggested that this is a case of severe overfitting. I made a neural network, which uses residual boosting (which is done via a KNN), and I am ...
0
votes
0
answers
24
views
Why doesn't U-Net work with images different from the dataset?
I have implemented a U-Net, similar to this implementation, but for a different dataset, this one, to segment roads.
It works fine using the test folder images, but, for example, when I pick a print ...
- 101
1
vote
0
answers
61
views
Is there a way, while training (with contrastive learning) the embedding network, to find the test accuracy?
I aim to do action recognition in videos on a private dataset.
To compare with the existing state-of-the-art implementations, other guys published their code on Github, like the one here (for the ...
1
vote
1
answer
101
views
How to build a test set for a model in industry?
Most of the tutorials only teach us to split the whole dataset into three parts: training set, develop set, and test set. But in the industry, we are kind of doing test-driven development, and what ...
- 877
3
votes
1
answer
83
views
How do I select the (number of) negative cases, if I'm given a set of positive cases?
We were given a list of labeled data (around 100) of known positive cases, i.e. people that have a certain disease, i.e. all these people are labeled with the same class (disease). We also have a much ...
- 33
1
vote
0
answers
49
views
Wouldn't training the model with this data lead to inaccuracies since the testing data would not be normalized in a similar way?
I was trying to normalize my input data images for feeding to my convolutional neural network and wanted to use standardize my input data.
I referred to this post, which says that ...
- 11
3
votes
1
answer
2k
views
What is the reason behind using a test batch size?
If one examines the SSD: Single Shot MultiBox Detector code from this GitHub repository, it can be seen that, for a testing phase (evaluating network on test data set), there is a parameter ...
- 143
1
vote
1
answer
851
views
How to perform PCA in the validation/test set?
I was using PCA on my whole dataset (and, after that, I would split it into training, validation, and test datasets). However, after a little bit of research, I found out that this is the wrong way to ...
- 121
4
votes
1
answer
194
views
What are "development test sets" used for?
This is a theoretical question. I am a newbie to artificial intelligence and machine learning, and the more I read the more I like this. So far, I have been reading about the evaluation of language ...
- 143