Questions tagged [training-datasets]
For questions related to the dataset used to train machine learning models, such as neural networks. The training dataset is different from the validation and test datasets, which are used for early stopping (and/or hyperparameter optimization) and to test the final model's performance, respectively.
56
questions
2
votes
1
answer
76
views
Why are some LLMs trained on both CommonCrawl and Wikipedia/StackExchange?
Some LLMs are trained on both CommonCrawl and Wikipedia/StackExchange. Why? Does CommonCrawl already contain Wikipedia/StackExchange?
E.g., from the LLaMa 1 paper:
and from https://huggingface.co/...
- 2,804
0
votes
0
answers
4
views
Individual Data point size within a dataset for finetuning for translation
I need to fine tuning a LLM for a translation task for specific domain. Could someone please advise if length of each data point within the data set has any impact on eventual performance of the LLMs. ...
0
votes
1
answer
27
views
How to label missing/default values for a named entity recognition dataset
I am building the training dataset for a named entity recognition model, with 2 tags: Name and Category and I am using a pre-...
- 3
1
vote
1
answer
140
views
What causes my loss curve to consistently oscillate when training an LLM?
Why is my loss curve consistently oscillating? Every 50 steps it jumps back up. I'm assumming there's a bug in my data, since I'm using this colab notebook that shows a proper train/loss at the bottom....
- 11
1
vote
0
answers
39
views
Is there a neural network architecture specialized for mapping lower-to-higher-dimensional data?
I am building a neural network that takes in a set of 86 parameters (primarily architecture-related, such as building floor area, kitchen size, number of a certain type of furniture, etc.) and outputs ...
- 143
-1
votes
1
answer
48
views
Open access Adept-like dataset? (LLM-to-computer-input) [closed]
Here's a demo for Adept ACT-1 for Transformers. I don't doubt that one could create a demo video using zero-shot; actually I tested just now and the basic chat.openai.com interface was able to do some ...
- 146
2
votes
1
answer
170
views
What's the architecture and size of neural-network-based reward models as used in reinforcement learning by human feedback
My rough understanding of RLHF as used for ChatGPT in a nutshell is this:
A reward model is trained using comparisons of different responses
to the same prompt. Human trainers rank these responses ...
0
votes
0
answers
11
views
Can I combined the trained model between different source but same model structure?
Here I got two different deep-learning models that were trained by LSTM and time-series data.
The data is the usage percentage of CPU from two different computers.
Each computer job was the same. It ...
- 103
2
votes
2
answers
71
views
0
votes
1
answer
40
views
AI and Machine Learning Prediction Algorithms for predicting outcome results of Hypothetical poll
Can artificial intelligence and Machine Learning Prediction Algorithms assist in deciding the Outome Results of a Hypothetical Online Poll?
Poll: Selecting favorite American President till date.
...
- 139
0
votes
0
answers
99
views
Can AI be used for submarine detection?
Recently a lot of work has been done in connection to using AI in the search for extraterestrial life, the SETI project.
Could the same methodology be used by the military, in order to detect ...
0
votes
0
answers
24
views
Longer DNN training times when using evolutionary algorithms
I am comparing my deep neural network (DNN) performance when using 2 types of optimizers: gradient-based Adam (properly tuned) and a population-based optimization algorithm (e.g., genetic algorithm (...
1
vote
1
answer
61
views
Creating a Dataset from Time Series Data
Context
I'd like to build a regression model for this data to predict a user's test scores given their study habits.
Basically, the variables are in two separate csv tables similar to the ones below. ...
- 113
0
votes
1
answer
39
views
What papers can I read that explore model performance vs dataset size?
I am trying to estimate how many images I need to label for an object detection task. I understand a lot of variables are at play, but I'd like to find some papers that have already explored this ...
0
votes
0
answers
283
views
When would you use prompting vs. fine tuning?
I would like to hear your thoughts on when is it appropriate to use prompting vs. fine-tuning.
Does one make more sense for specific tasks than the other.
Kindly elaborate.
0
votes
2
answers
69
views
Why does linearly decreasing batch sizes result in exponentially increasing training times?
I'm quite new to machine learning and wanted to ask a question regarding why reducing batch sizes cause exponentially increasing training times. An example of this relationship between batch size and ...
- 3
0
votes
0
answers
28
views
Effectiveness of DNN training with reduced Batch randomness
So here's an example set to help explain my doubt. Suppose I have 80,000 total images available for a DNN training task. With a batch size of 32, that is 2500 batches.
Now let's say I partition the ...
6
votes
1
answer
2k
views
How was ChatGPT trained?
I know that large language models like GPT-3 are trained simply to continue pieces of text that have been scraped from the web. But how was ChatGPT trained, which, while also having a good ...
- 313
9
votes
1
answer
1k
views
What causes ChatGPT to generate responses that refer to itself as a bot or LM?
ChatGPT occasionally generates responses to prompts that refer to itself as a "bot" or "language model."
For instance, when given a certain input (the first paragraph of this ...
- 193
0
votes
0
answers
23
views
What are the best ways to preprocess landmarks to train a Neural Network?
I'd like to know, generally speaking, which are the most useful ways to preprocess landmarks to use as training set for a Deep Neural Network. Since they're put in a 3D space, would that be enough ...
- 1
1
vote
0
answers
168
views
Is it possible to create an AI that produces output without giving it input?
AFAIK an AI is first trained using a data set of input and output values.
After the training proccess you can give the AI input and it will produce output.
For example when you write a sentence to an ...
- 119
1
vote
1
answer
3k
views
Fine-tune GPT-Neo with prompt and completion?
I'm new to AI and machine learning.
To fine-tune GPT-3, I understand that we need a set of training examples that each consist of a single input ("prompt") and its associated output ("...
- 111
1
vote
0
answers
28
views
bad prediction when having noise on the data: LSTM time-series regression
I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
- 11
3
votes
4
answers
1k
views
How is MNIST only providing the training and the test sets? What about the validation?
I was taught that, usually, a dataset has to be divided into three parts:
Training set - for learning purposes
Validation set - for picking the model which minimize the loss on this set
Test test - ...
- 157
1
vote
1
answer
265
views
Batching together similar length sequences to avoid padding and packing
I am training an RNN in PyTorch to produce captions for images. It's a pretty standard architecture – the image is processed by a pre-trained InceptionV3 to extract features, the recurrent module ...
- 111
0
votes
1
answer
34
views
What kind of neural network and GPU should I use to classify images into > 10 000 classes?
I am trying to developp an image classifier that would have more than 10 000 classes but I don't know what kind of neural network I should use ?
Some Other questions arise from this one :
How big ...
0
votes
0
answers
42
views
Is the neural network 100% accurate on training data if epoch loss is minimized to 0?
This seems like a silly, trivial question, but I just want to confirm it in case I'm missing something.
I'm trying to train a ReLU neural network, which is supposed to be a function that satisfies ...
- 111
0
votes
1
answer
86
views
Storing training dataset in a platform like mlflow
I am pretty new to Machine learning and would like to know whether there are experiment management platforms that also allow storing and managing training datasets (images, in my case). I am familiar ...
- 181
1
vote
1
answer
94
views
Datasets input at model.fit produce unexpected results of training loss vs validation loss
Im trying to train a neural network (VAE) using tensorflow and Im getting different results based on the type of input in the model.fit.
When I input arrays I get normal difference between the ...
user56546
1
vote
0
answers
12
views
how to manage the impact of Covid on building a machine learning model
I need your suggestions for using historical data to build a machine learning model for analyzing the market and build an AI model(tree based model/random forest or regression analysis) for setting ...
- 111
1
vote
1
answer
131
views
Should I include overlapping (input) Data in my training data
If I have time dependent data and want to predict the relative change for a future time. Should I separate the data so that the input times don't overlap?
With an example:
I have hourly temperature ...
- 13
0
votes
0
answers
48
views
How to use oxford5k for training?
Generally, we have training data with landmark IDs, their GTs (positive samples), and then separate query images and corresponding positive samples for evaluation.
In the Oxford5k or ROxford5k, one ...
- 111
1
vote
0
answers
69
views
Why does the SVM perform poorly on test data that has a different class distribution than the training data?
Do you know why the SVM performs poorly on test data that has a different class distribution than the training data? The training data has around 15 classes, and the additional testing data has around ...
- 11
0
votes
1
answer
49
views
How to create a dataset for binary classification
I would like to classify whether a pot of water is boiling or not using a CNN.
Is it enough to take pictures of boiling water using only one pot, or should I use different pots for this to generalize ...
- 123
0
votes
1
answer
78
views
Is there a standard term for the following flaw in the data?
I wonder if following characteristic of data has some standard "professional" or scientific term associated with it.
Let's assume that I have a set of dog/cat images labeled 0 for a cat and ...
- 1,240
0
votes
1
answer
77
views
Why "large set of training data" is needed in Neural Network AI training?
I often heard people saying, "large set of training data is needed for producing an accurate AI".
But when I looked for articles explaining backpropagations online, it all seems like you ...
- 11
2
votes
2
answers
410
views
How many unique angles of an object do you need in your image training set in order to correctly classify it?
I'm interested in using ResNet-50 to classify images of objects for around 1000 unique classes. I'm wondering if there is any way to estimate how many unique angles I need in my training set to ...
- 145
5
votes
1
answer
619
views
How can I estimate how many photos I need to train ResNet-50 for image classification?
I am working on a project where I have to classify around 1000 unique objects. I'm trying to plan how much training data I will need to collect. I was planning on using ResNet-50. Is there anyway I ...
- 145
0
votes
1
answer
66
views
Given a dataset of people with and without cancer, should I split it into training and test datasets such that the same person is not in both?
I have a database that contains healthy persons and lung cancer patients. I need to design a deep neural network for the binary classification problem (cancer/no cancer). I need to split the dataset ...
- 111
5
votes
2
answers
757
views
Do we need automatic hyper-parameter tuning when we have a large enough dataset?
Hyperparameter tuning is the process of selecting the optimal hyperparameters for an ANN.
Now, my guess is that, if we have sufficient data (say, 1.4 million for, say, 6 features), the model can be ...
- 299
1
vote
0
answers
35
views
Should I train a neural network with data with or without a constraint?
I want to train a Neural Network (NN) using a dataset. I want to use the NN model as a prediction function in one algorithm. However, in the algorithm, any data that does not meet a specific ...
0
votes
1
answer
39
views
Does the ANN's training data include the proper output for every neuron?
I was designing an Artificial Neural Network a while back, but hit a bump when I got to the backpropagation. I was having trouble making the script choose whether to add or subtract from the weights, ...
- 137
2
votes
2
answers
73
views
Why not make the training set and validation set one if their roles are similar?
If the validation set is used to tune the hyperparameters and the training set adjusts the weights, why don't they be one thing as they have a similar role, as in improving the model?
- 43
2
votes
3
answers
2k
views
Can people use neural networks without providing the set of training data?
It seems that neural networks (NNs) can be applied to supervised learning, unsupervised learning and reinforcement learning. Some people even train neural networks without the set of training data. If ...
0
votes
3
answers
185
views
Is binary classification using CNN possible if the training data only consists of one class?
Is binary classification using CNN possible if the training data only consists of one class?
I am working on landslide risk assessment using Convolutional Neural Networks and I want to train a network ...
- 1
1
vote
2
answers
157
views
Is creating dataset only by augmentation a bad practice?
I wonder if creating data set only by augmentation base images is a bad practice.
I mean the situation when you have to train net to predict really simple patterns, for example printed-like digits. ...
- 242
1
vote
1
answer
51
views
How to source training data in ML for information security?
A company entrusts a Data Scientist with the mission of processing and valuing data for the research or treatment of events related to traces of computer attacks. I was wondering how would he get the ...
1
vote
1
answer
430
views
How to split data for meta-learning?
I've been trying to understand the meta-learning paradigm, more precisely, the optimization-based models, such as MAML, but I have a hard time understanding how I should correctly split my data to ...
- 1,108
0
votes
0
answers
24
views
Why doesn't U-Net work with images different from the dataset?
I have implemented a U-Net, similar to this implementation, but for a different dataset, this one, to segment roads.
It works fine using the test folder images, but, for example, when I pick a print ...
- 101
3
votes
1
answer
83
views
How do I select the (number of) negative cases, if I'm given a set of positive cases?
We were given a list of labeled data (around 100) of known positive cases, i.e. people that have a certain disease, i.e. all these people are labeled with the same class (disease). We also have a much ...
- 33