Questions tagged [datasets]
For questions related to sets of data and their use in AI.
266
questions
0
votes
0
answers
12
views
i need dataset for human activity and diseases from smartphone accelerometer sensor?
project aims to develop a health monitoring system utilizing smartphone accelerometer sensors and machine learning techniques. The primary focus is on detecting specific health conditions through the ...
1
vote
3
answers
62
views
Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?
Imagine we have some sort of "next token predictor," either with transformer architecture, LSTM, or just a HMM (though the terminology I use here will be less aligned to HMMs, I believe the ...
- 113
0
votes
0
answers
17
views
Is synthetic data just a placebo for immature models?
I apologize for the provocative question, but let me elaborate.
I am trying to wrap my head around the logic of synthetic data.
When you train a model what you are trying to do is to teach the ground ...
- 295
0
votes
0
answers
9
views
Could you recommend action recognition dataset?
background
I want to do action recognition for academic, but I can't find dataset what I want.
so, I wonder Which dataset is suitable for me...
What I want?
I want to find dataset such as..
labeling ...
- 27
0
votes
0
answers
14
views
Training YOLO on YOLO results
I have a large unlabelled dataset. What if I use YOLO to label it? Will this dataset be useful to train a better YOLO model? What if I then finetune it on a smaller labelled dataset? My usecase ...
0
votes
1
answer
28
views
What are the differences between loss surfaces that "derive"from different observations?
If I understand right that each observation whithin a dataset, creates a different loss surface where we want to find the global minimum.
How different those surfaces one from another?
Would it be ...
- 181
0
votes
1
answer
411
views
What role does data quality plays in the LLM scaling laws?
DeepMind released the Training Compute-Optimal Large Language Models paper in 2022 which describe some scaling laws for LLMs. As far as I understand this is the most accredited reference to estimate ...
- 123
0
votes
0
answers
13
views
Datasets for conversation topic classification with focus on chat
Are there good datasets or even pretrained models for classifying chat lines? I'd like to infer the topic people are talking about and possibly topic changes.
One of the problems with data that is ...
- 280
0
votes
1
answer
40
views
AI and Machine Learning Prediction Algorithms for predicting outcome results of Hypothetical poll
Can artificial intelligence and Machine Learning Prediction Algorithms assist in deciding the Outome Results of a Hypothetical Online Poll?
Poll: Selecting favorite American President till date.
...
- 139
1
vote
1
answer
61
views
Creating a Dataset from Time Series Data
Context
I'd like to build a regression model for this data to predict a user's test scores given their study habits.
Basically, the variables are in two separate csv tables similar to the ones below. ...
- 113
0
votes
0
answers
13
views
Is there a deep learning model when train data and test data (or real data) have some common and uncommon labels?
I am building DL model that predict test data (or real) that has some different labels compared to training dataset.
My semi-supervised model is kind of terrible to predict test data (77~85% ish), and ...
0
votes
1
answer
53
views
What are possible reasons for the validation loss increasing with more data?
I trained a neural network on an NLP problem and compared the loss and BLEU score on the validation data with the same training parameters in two scenarios: a) when I trained on 25% of the data, b) ...
1
vote
2
answers
65
views
Is data useless for a neural network if some inputs are derivatives of other inputs?
That is, if some of the inputs to a neural network can be calculated by a pre-determined function whose variables are other inputs, then are those specific inputs useless?
For example, suppose there ...
- 67
1
vote
0
answers
163
views
How to get ZINC 500k dataset?
I have been using the ZINC graph regression dataset available through pytorch geometric datasets for a while now in two of its modes (12k examples and 250k examples). However, in the PapersWithCode ...
- 201
0
votes
0
answers
33
views
How to organize data for training a neural network to raytrace images
I am trying to train a neural network to go from a series of numbers(data about a 3D scene, such as camera position, sphere position, radius, and color, etc) to a raytraced image. I can generate as ...
- 1
1
vote
1
answer
143
views
How can imitation learning data be collected?
How can imitation learning data be collected? Can I use a neural network for that? It might be noisy. Should I use manual gathering?
- 862
1
vote
1
answer
3k
views
Fine-tune GPT-Neo with prompt and completion?
I'm new to AI and machine learning.
To fine-tune GPT-3, I understand that we need a set of training examples that each consist of a single input ("prompt") and its associated output ("...
- 111
0
votes
0
answers
35
views
Do Neural Networks tend to have Zero Mean Errors in each Output?
My NN (a few linear layers with ReLUs + batch normalization, no activation in the last layer) learns to approximate vector-valued labels $y_z$ from data $z\sim\rho_z$ in a supervised way, i.e. net$(z)=...
- 101
0
votes
1
answer
849
views
Split dataset into Train/Validation/Test for Object Detection
I have a dataset for Object Detection with YOLO format labels, each imagine can have occurences of different classes and multiple occurences of the same class.
How can the dataset be divided into ...
10
votes
2
answers
2k
views
How can I encode angle data to train neural networks?
I am training a neural network where the target data is a vector of angles in radians (between $0$ and $2\pi$).
I am looking for study material on how to encode this data.
Can you supply me with a ...
- 299
1
vote
1
answer
61
views
Can independent datasets be artificially combined for multimodal learning (semi-synthetic data generation)?
BACKGROUND: To apply multimodal machine learning (ML), the various data modalities typically come from the same example (e.g., chest X-ray (modality 1) and ...
- 922
2
votes
1
answer
65
views
Are the "artifacts" in select Keras MNIST training images really there or is my download corrupt?
I'm having fun with a ludicrously well known and used dataset: mnist.
I am doing it with a huge and well known tool: keras.
Please excuse the red dots, something else I was doing. I have otherwise ...
- 371
0
votes
1
answer
167
views
Why does data augmentation using synthetic data generated by one model improve the performance of another model?
I understand from articles like this one that synthetic data generated by one model based on real data can improve the performance of a second model. Can anyone help me understand the intuition behind ...
0
votes
1
answer
110
views
Machine Learning Methods commonly used when data are scarse
It is well-known that deep neural networks require lots of data to perform reliably and well. A commonly-cited statistic is that you need at least 10,000 examples per class for a classification ...
-1
votes
1
answer
134
views
Generating synthetic time series data with limited data
I would like some opinions on my current situation.
I have a set of time series data that I want to forecast. The data however is not very long (around 500 rows) so I was looking into generating many ...
- 13
0
votes
1
answer
58
views
To train a mushroom vs. non-mushroom image classifier, which images should comprise the negative class?
I want to train a model that will identify if the image is mushroom and non-mushroom.
If the image is mushroom, I will use another existing model to categorize if its poisonous or edible.
I want to ...
3
votes
4
answers
1k
views
How is MNIST only providing the training and the test sets? What about the validation?
I was taught that, usually, a dataset has to be divided into three parts:
Training set - for learning purposes
Validation set - for picking the model which minimize the loss on this set
Test test - ...
- 157
2
votes
1
answer
203
views
Is there an image classification dataset where the class depends on spatial relations?
My question is pretty much the one asked above. To clarify a bit further: I have only found datasets that do object localization and that also have relations between the objects annotated (like: "...
- 21
1
vote
0
answers
21
views
Can I use a dataset with real-world images and corresponding actions that the expert took to train an IRL algorithm?
Offline Reinforcement Learning approaches like Inverse Reinforcement Learning/ Batch RL/ imitation learning/ behavior cloning allow us to use previous demonstrations by an expert to learn a policy.
...
- 11
1
vote
2
answers
760
views
Does more data increase training accuracy in neural networks?
I was wondering what is the performance benefit of feeding more data to a machine learning model like a neural network? Like I know one of the benefits is that it increases generalization - testing ...
- 11
1
vote
1
answer
263
views
Feature Engineering on transactional dataset clustering
I have a data set with transactions details from different business (roughly 1 thousand business entities). Each row is a transaction. The structure of the dataset is as follows:
client_id
Sex
Age
...
1
vote
2
answers
72
views
Master theorem about polynomial classifiers?
Does anyone know if there is a theorem or counterexample establishing whether or not for any given binary classification task in some finite (possibly large) dimensional vector space of attributes, ...
0
votes
1
answer
91
views
Meaning of Large Dataset for machine learning
Some online answers about parameters in machine learning mention that it is dependent on the size of dataset we have (if it is a large dataset or not). Is this size related to the number of samples we ...
0
votes
1
answer
195
views
combine two features in dataset?
I have a data set containing the number of security gaps and the level of that gap for a specific website.
Now suppose I have 2 features in this data set, the first feature is the number of a ...
1
vote
1
answer
94
views
Datasets input at model.fit produce unexpected results of training loss vs validation loss
Im trying to train a neural network (VAE) using tensorflow and Im getting different results based on the type of input in the model.fit.
When I input arrays I get normal difference between the ...
user56546
1
vote
1
answer
132
views
Should I include overlapping (input) Data in my training data
If I have time dependent data and want to predict the relative change for a future time. Should I separate the data so that the input times don't overlap?
With an example:
I have hourly temperature ...
- 13
0
votes
1
answer
68
views
What should be taken as random variables in the distributions of datasets?
Consider the following two paragraphs taken from the paper titles Generative Adversarial Nets by Ian J. Goodfellow et.al
#1: Abstract
We propose a new framework for estimating generative models via ...
- 3,612
1
vote
2
answers
215
views
What makes a 'good' dataset
for the usage of ML technologies, having a appropriate dataset is arguably the first and fundamental step one has to tackle by either aquiring a dataset from external sources or creating their own.
...
- 11
0
votes
1
answer
66
views
Is there a way to improve the low-quality data?
I'm on a robotics team and we've been tasked to write a program to differentiate between a live and dead fish. We've been given ~15 minutes of training footage and it's absolutely terrible. It's low ...
user55176
7
votes
2
answers
790
views
Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?
I plan to use my predictions as ground truth to continue training my model. These predictions are of course reviewed during this process. Is there an argument against that (reinforcement of slight ...
- 73
0
votes
0
answers
19
views
Which existing model could be used for wind speed and direction prediction?
I am trying to predict the wind speed and wind direction in a graph network for a geographical area. The dataset includes the start and end nodes, the distance between them, and wind speed and ...
- 1
4
votes
1
answer
261
views
What is the total number of actions and rewards count
Reading this two articles about Reinforcement Learning:
Deep Reinforcement Learning with Double Q-learning by Hado van Hasselt et al.
Human-level control through deep reinforcement learning by ...
- 143
0
votes
0
answers
48
views
How to use oxford5k for training?
Generally, we have training data with landmark IDs, their GTs (positive samples), and then separate query images and corresponding positive samples for evaluation.
In the Oxford5k or ROxford5k, one ...
- 111
0
votes
3
answers
210
views
How to deal with an unbalanced dataset?
I'm constructing a feed forward neural network that predicts whether a patient will get a stroke or not. However, my dataset is very unbalanced. Out of 5111 rows, 250 contain patients that have had a ...
- 9
0
votes
1
answer
49
views
How to create a dataset for binary classification
I would like to classify whether a pot of water is boiling or not using a CNN.
Is it enough to take pictures of boiling water using only one pot, or should I use different pots for this to generalize ...
- 123
0
votes
1
answer
85
views
Having the negative cases in the same batch vs. shuffling the dataset
I am working on a model for an NLP task. The model encodes the text and has a regression output layer.
In this task, from each instance (positive), I create several negative cases using a specific ...
- 123
0
votes
1
answer
78
views
Is there a standard term for the following flaw in the data?
I wonder if following characteristic of data has some standard "professional" or scientific term associated with it.
Let's assume that I have a set of dog/cat images labeled 0 for a cat and ...
- 1,240
0
votes
0
answers
29
views
What are the "per image" annotations that are generally used for image datasets in AI?
Computer vision is highly benefited by AI algorithms. Image data is abundantly available. There are different varieties of tasks such as image classification, prediction, segmentation, generation, ...
- 3,612
0
votes
0
answers
40
views
What is all necessary types of data for a bidirectional RNN to learn embeddings?
Bidirectional RNNs are used for generating the semantic vectors of the text at the sentence level and word level.
In order to train a CNN for the classification tasks, images, and labels/outputs are ...
- 3,612
1
vote
0
answers
21
views
What are the types of data in which the order of instances does matter?
In general, the order of instances in the datasets that are used in machine learning is immaterial. But there are exceptions. Timeseries data is one such exception I know. Consider the following two ...
- 3,612