Questions tagged [training-datasets]

For questions related to the dataset used to train machine learning models, such as neural networks. The training dataset is different from the validation and test datasets, which are used for early stopping (and/or hyperparameter optimization) and to test the final model's performance, respectively.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

How to use oxford5k for training?

Generally, we have training data with landmark IDs, their GTs (positive samples), and then separate query images and corresponding positive samples for evaluation. In the Oxford5k or ROxford5k, one ...
user avatar
  • 101
1 vote
0 answers
38 views

Why does the SVM perform poorly on test data that has a different class distribution than the training data?

Do you know why the SVM performs poorly on test data that has a different class distribution than the training data? The training data has around 15 classes, and the additional testing data has around ...
user avatar
  • 11
0 votes
0 answers
15 views

When generating segmentation mask, is it better for the ground truth mask to be a bit inside the object than outside?

I got asked this question today, and I was wondering. When manually annotating images for ground truth, is it better for the model to get segmentation masks that are a bit inside the object or a bit ...
user avatar
  • 101
0 votes
0 answers
41 views

Training Variational Autoencoder (VAE) on custom dataset

I am training a VAE on a custom dataset for anomaly detection. The data consists of around 500 images of empty white boxes (at different positions) such as below: Original Image of empty box I am ...
user avatar
-1 votes
1 answer
25 views

How to create a dataset for binary classification

I would like to classify whether a pot of water is boiling or not using a CNN. Is it enough to take pictures of boiling water using only one pot, or should I use different pots for this to generalize ...
user avatar
0 votes
1 answer
69 views

Is there a standard term for the following flaw in the data?

I wonder if following characteristic of data has some standard "professional" or scientific term associated with it. Let's assume that I have a set of dog/cat images labeled 0 for a cat and ...
user avatar
  • 1,029
0 votes
1 answer
66 views

Why "large set of training data" is needed in Neural Network AI training?

I often heard people saying, "large set of training data is needed for producing an accurate AI". But when I looked for articles explaining backpropagations online, it all seems like you ...
user avatar
  • 11
2 votes
1 answer
39 views

How many unique angles of an object do you need in your image training set in order to correctly classify it?

I'm interested in using ResNet-50 to classify images of objects for around 1000 unique classes. I'm wondering if there is any way to estimate how many unique angles I need in my training set to ...
user avatar
4 votes
1 answer
91 views

How can I estimate how many photos I need to train ResNet-50 for image classification?

I am working on a project where I have to classify around 1000 unique objects. I'm trying to plan how much training data I will need to collect. I was planning on using ResNet-50. Is there anyway I ...
user avatar
0 votes
1 answer
54 views

Given a dataset of people with and without cancer, should I split it into training and test datasets such that the same person is not in both?

I have a database that contains healthy persons and lung cancer patients. I need to design a deep neural network for the binary classification problem (cancer/no cancer). I need to split the dataset ...
user avatar
  • 111
4 votes
2 answers
698 views

Do we need automatic hyper-parameter tuning when we have a large enough dataset?

Hyperparameter tuning is the process of selecting the optimal hyperparameters for an ANN. Now, my guess is that, if we have sufficient data (say, 1.4 million for, say, 6 features), the model can be ...
user avatar
1 vote
0 answers
23 views

Should I train a neural network with data with or without a constraint?

I want to train a Neural Network (NN) using a dataset. I want to use the NN model as a prediction function in one algorithm. However, in the algorithm, any data that does not meet a specific ...
user avatar
0 votes
1 answer
37 views

Does the ANN's training data include the proper output for every neuron?

I was designing an Artificial Neural Network a while back, but hit a bump when I got to the backpropagation. I was having trouble making the script choose whether to add or subtract from the weights, ...
user avatar
0 votes
0 answers
38 views

How big should the dataset for retraining ssd_mobilenet_v2 be?

I have retrained ssd_mobilenet_v2 using my own dataset with 2 classes (pen or pencil), using object detection API. For my project, I expect users to select specific pencils from all pencils and ...
user avatar
  • 1
0 votes
0 answers
14 views

Is it possible to train a model on files of code and output questions about it?

I want to know if it is feasible to use deep learning to generate homework questions for a course on logic. My input data of programming functions and desired output of respective homework questions ...
user avatar
  • 1
2 votes
2 answers
65 views

Why not make the training set and validation set one if their roles are similar?

If the validation set is used to tune the hyperparameters and the training set adjusts the weights, why don't they be one thing as they have a similar role, as in improving the model?
user avatar
1 vote
3 answers
186 views

Can people use neural networks without providing the set of training data?

It seems that neural networks (NNs) can be applied to supervised learning, unsupervised learning and reinforcement learning. Some people even train neural networks without the set of training data. If ...
user avatar
0 votes
3 answers
69 views

Is binary classification using CNN possible if the training data only consists of one class?

Is binary classification using CNN possible if the training data only consists of one class? I am working on landslide risk assessment using Convolutional Neural Networks and I want to train a network ...
user avatar
  • 1
1 vote
2 answers
87 views

Is creating dataset only by augmentation a bad practice?

I wonder if creating data set only by augmentation base images is a bad practice. I mean the situation when you have to train net to predict really simple patterns, for example printed-like digits. ...
user avatar
1 vote
1 answer
35 views

How to source training data in ML for information security?

A company entrusts a Data Scientist with the mission of processing and valuing data for the research or treatment of events related to traces of computer attacks. I was wondering how would he get the ...
user avatar
1 vote
1 answer
121 views

How to split data for meta-learning?

I've been trying to understand the meta-learning paradigm, more precisely, the optimization-based models, such as MAML, but I have a hard time understanding how I should correctly split my data to ...
user avatar
  • 1,078
0 votes
0 answers
20 views

Why doesn't U-Net work with images different from the dataset?

I have implemented a U-Net, similar to this implementation, but for a different dataset, this one, to segment roads. It works fine using the test folder images, but, for example, when I pick a print ...
user avatar
3 votes
1 answer
81 views

How do I select the (number of) negative cases, if I'm given a set of positive cases?

We were given a list of labeled data (around 100) of known positive cases, i.e. people that have a certain disease, i.e. all these people are labeled with the same class (disease). We also have a much ...
user avatar
  • 33
6 votes
1 answer
123 views

During neural network training, can gradients leak sensitive information in case training data fed is encrypted (homomorphic)?

Some algorithms in the literature allow recovering the input data used to train a neural network. This is done using the gradients (updates) of weights, such as in Deep Leakage from Gradients (2019) ...
user avatar
  • 73
2 votes
0 answers
249 views

How does one continue the pre-training in BERT?

I need some help with continuing pre-training on Bert. I have a very specific vocabulary and lots of specific abbreviations at hand. I want to do an STS task. Let me specify my task: I have domain-...
user avatar
5 votes
0 answers
2k views

How many training data is required for GAN?

I'm beginning to study and implement GAN to generate more datasets. I'll just try to experiment with state-of-the-art GAN models as described here https://paperswithcode.com/sota/image-generation-on-...
user avatar
  • 195
4 votes
1 answer
1k views

What happens to the training data after your machine learning model has been trained?

What happens after you have used machine learning to train your model? What happens to the training data? Let's pretend it predicted correct 99.99999% of the time and you were happy with it and wanted ...
user avatar
  • 159
2 votes
2 answers
1k views

What is the effect of training a neural network with randomly generated fake data that satisfies certain constraints?

I have a neural network with 2 inputs and one output, like so: ...
user avatar
  • 123
4 votes
1 answer
86 views

What are "development test sets" used for?

This is a theoretical question. I am a newbie to artificial intelligence and machine learning, and the more I read the more I like this. So far, I have been reading about the evaluation of language ...
user avatar