Questions tagged [datasets]
For questions related to sets of data and their use in AI.
164
questions
0
votes
2answers
21 views
What is the meaning of test data set in naive bayes classifier or decision trees?
What is the beneft of a test data set, especially for naive bayes estimator or decision tree construction?
When using a naive bayes classifier the probabilities are a fact. As far as I know there is ...
0
votes
0answers
8 views
How can I convert a Moses dataset, which is already divided into training and test datasets, to two files? [migrated]
I have a moses dataset that is already divided into training and test datasets, but it is all in one CSV file, and I want to break it up into different variables (train, test).
Here are the first few ...
2
votes
1answer
33 views
What are “proxy data sets” in machine learning?
The paper Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration uses the term "proxy data sets" in this way
To develop DL ...
3
votes
2answers
72 views
Why do we need both the validation set and test set?
I know that this has been asked a hundred times before, however, I was not able to find a question (and an answer) which actually answered what I wanted to know, respectively, which explained it in a ...
0
votes
2answers
24 views
What does it mean to have epochs=30 in Keras' fit method given certain data?
I have read a lot of information about several notions, like batch_size, epochs, iterations, but because of explanation was without numerical examples and I am not native speaker, I have some kind of ...
1
vote
0answers
13 views
Will structured knowledge bases continue to be used in question answering with the likes of BERT gaining popularity?
This may come across as an open and opinion-based question, I definitely want to hear expert opinions on the subject but I am also looking for references to materials that I can read deeply.
One of ...
0
votes
0answers
21 views
How should I clean the columns of the dataset “Application-Layer DDoS Dataset” from Kaggle? [migrated]
I'm trying to get the right kind of data to build my model. I'm using TensorFlow's DNN classifier for this model. The dataset I'm using is Application-Layer DDoS Dataset from Kaggle. I've encoded the ...
2
votes
0answers
19 views
Boston Housing Current Best Model
I'm currently looking for a standard data-set to test some new regression models I've been developing on, and the "Boston Housing Prices" data-set seemed to stand out, since it's very standard.
...
3
votes
0answers
39 views
How should I select the features for predicting diseases?
My aim is to create a trained model for predicting diseases. Now diseases are classified based on the following criteria in general:
Causes of the Disease
Pathogenesis (the mechanism by which the ...
0
votes
0answers
21 views
Creating custom sequential dataset in keras [migrated]
I have data saved in multiple csv files (16000+) and my aim is to create a dataset from these files. Each csv has the columns: "Source node", "Path", "Target node". In each of these columns is a one-...
1
vote
2answers
46 views
Validation loss is lower than Training loss
I am training a classifier to identify 24 hand signs of American Sign Language. I created a custom dataset by recording videos in different backgrounds for each of the signs and later converted the ...
1
vote
1answer
45 views
Are sentences from the same document independent and identically distributed?
I am trying to build an LSTM model to generate Shakspeare-like poems. I have data set $\{s_1, s_2, \dots, s_m \}$, which are sentences of Shakespeare poems, and each sentence contains words $\{w_1, ...
1
vote
1answer
40 views
Image dataset for pomegranate plant disease
I am implementing a project on pomegranate plant disease in Machine learning. I want a dataset of all kind images of a healthy and unhealthy part of the pomegranate plant. I got a dataset from ...
3
votes
1answer
60 views
Can you let specific data impact a neural network more than other data?
I have a lot of empty values in my dataset, so I want to let my neural network 'learn more' on the rows that have no empty values because these rows are of higher importance.
Is there a way to do ...
3
votes
1answer
47 views
Is a basic neural network architecture better with small datasets?
I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use.
Will I get more accurate results when I use a small ...
2
votes
2answers
83 views
Is there a possibility that there is no relationship between some inputs and outputs?
I'm doing machine learning projects. I took a look at many datasets I worked with, mostly there are already famous datasets that everyone uses.
Let's say I decided to make my own dataset. Is there a ...
1
vote
1answer
34 views
Using three image datasets with different image sizes to train a CNN
I've just started with AI and CNN networks.
I have two NIFTI images dataset, one with (240, 240) dimensions and the other one with (256, 132). Each dataset is front a different hospital and machine.
...
2
votes
1answer
28 views
How to do machine translation with no labeled data?
Can it be possible to train a neural network with no parallel bilingual data?
1
vote
1answer
27 views
Is it legal to construct a public image database (for deep learning) with images from the internet? [closed]
I am trying to put together a public agricultural image database of corn and soybeans, to train convolutional neural networks. The main method of image collection will be through taking pictures of ...
1
vote
0answers
27 views
Weird border artifacts when training a CNN
I've been trying to use this DeepLabv3+ implementation with my dataset (~1000 annotated images of the same box, out of the same video sequence): https://github.com/srihari-humbarwadi/...
0
votes
0answers
13 views
augmentation, cross-validation, test evaluation
I have, say, a (balanced) data-set with 2k images for binary classification. What I have done is that
randomly divided the data-set into 5 folds;
copy-pasted all 5-fold data-set to have 5 exact ...
2
votes
1answer
40 views
On which data evaluate an object detection model ? (similar or real life data ?)
I'm training an object detection model (SSD300) to detect and classify body poses in thermal images.
Even I have more than 2k different poses, but the background does not change much (I have only 5 ...
2
votes
3answers
105 views
Small size datasets for object detection, segmentation and localization
I am looking for a small size dataset on which I can implement object detection, object segmentation and object localization.
Can anyone suggest me a dataset less than 5GB? Or do I need to know ...
1
vote
0answers
23 views
Labeling for multilabel image classification
with a friend of mine, we got in an argument over how to label images for multi-label.
Note:
Groups of a species and the species of catfish is important to recognize.
The labels are:
'I': an ...
1
vote
1answer
33 views
Where can I upload a large photo database for public access? [closed]
I am applying for a grant, and one of the tasks we are seeking funding for is to make a large image database publicly available for users to train artificial intelligence (convolutional neural network)...
1
vote
0answers
12 views
EEG and Accelerometer Neural Network
I have frequency EEG data from fall and non-fall events and I am trying to incorporate it with accelerometer data that was collected at the same time. One approach is, of course, to use two separate ...
0
votes
0answers
22 views
How to use Before / After images to train a model
I am trying to create a model that can clean pictures of noise, blur, high luminosity etc, but I do not know how to do that. I have tried to search for it a lot, and I couldn't find anything that ...
2
votes
0answers
35 views
How can I compare EEG data with accelerometer data in 1 algorithm?
I have frequency EEG data from fall and non-fall events and I am trying to incorporate it with accelerometer data that was collected at the same time. One approach is, of course, to use two separate ...
0
votes
0answers
8 views
Hashing images detects false duplicates
Like I said in the title, I used a code I found to identify duplicate images however there are many false alarms. Can someone give me a link to some code I could use that really works? The problem ...
2
votes
0answers
86 views
Suggestion for finding the stable regions in spiral galaxy data?
I am working with a data set that consists of the actual pitch angle (given as PA(Y)) and the pitch angle at each radii (listed from 1 to 217). In the image below, ...
0
votes
3answers
55 views
How to correctly label images for multi-label classification?
I have images that contain lots of elements. Some I know, some I don't. I want to know if it's ok to only label those I do know. Let's take this image for example. I would label the green stuff and ...
1
vote
1answer
14 views
Ambulance dataset needed
Could I get a dataset that can classify ambulances?
I have searched everywhere, but, couldn't seem to get hold of a set of annotated images for ambulances.
1
vote
0answers
24 views
Should I add some noise when the dataset is small?
I want to create a small dataset (about 10 classes and 20-30 images each), should I add some noise (wrong label sample) in the training, validation and test datasets, and why?
0
votes
0answers
41 views
Is there an audio dataset with the corresponding phonemes in the audio?
I am looking for a dataset of clear audio, a corresponding transcript (optional), and most importantly a list of all the phonemes said in the audio, with the length of each phoneme and a mention of ...
1
vote
0answers
25 views
Are there free and easy-to-use annotation tools for 3D bounding boxes?
Are there free and easy-to-use annotation tools for 3D bounding boxes?
There is a lot of annotation tools out there, but most of them only good for 2d boxes.
0
votes
1answer
30 views
Is it possible to use AI for detecting the volume of a cup
I was just wondering if it's possible to use Machine Learning to train a model from a dataset of images of cups with a given volume in the image and then use object detection to detect other cups and ...
2
votes
0answers
129 views
How can I train YOLO with the COCO dataset?
I am trying to implement the original YOLO architecture for object detection, but I am using the COCO dataset. However, I am a bit confused about the image sizes of COCO. The original YOLO was trained ...
1
vote
0answers
244 views
How many training data required for GAN?
I'm beginning to study and implement GAN to generate more dataset. I'll just try to experiment with state-of-the-art GAN models as described in here https://paperswithcode.com/sota/image-generation-on-...
2
votes
2answers
46 views
Are there any easy ways to create annotated training images for object detection?
For the purposes of object detection, are there any easy ways to create annotated training images? For example, if we have $10,000$ images and want to draw bounding boxes on 2 objects for each image, ...
1
vote
0answers
16 views
learning object recognition of primitive shapes through transfer learning problem
Question on transfer learning object classification (MobileNet_v2 with 75% number of parameters) with my own synthetic data:
I made my own dataset of three shapes: triangles, rectangles and spheres. ...
2
votes
1answer
29 views
What does top N accuracy mean?
Places205-VGG, a CNN trained model for 205 scene categories of Places Database with 2.5 million images Places205 dataset has ...
3
votes
1answer
51 views
How could decision tree learning algorithms cope with imbalanced classes?
Decision trees and random forests may or not be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article Using Random Forest to ...
0
votes
0answers
53 views
Focal loss for imbalanced multi class classification in Pytorch
I want an example code for Focal loss in PyTorch for a model with three class prediction. My model outputs 3 probabilities.
...
1
vote
1answer
29 views
If an image contains two distinct objects, should I create a copy of this image with distinct labels for each copy?
Suppose we want to detect whether an object is one of the following classes: $\text{Object}_1, \text{Object}_2, \text{Object}_3$ and $\text{Person}$. Should the annotated images only contain bounding ...
1
vote
1answer
44 views
How can I merge two datasets? [closed]
I want to merge 2 data sets in one, but don't know the right approach to do it. The datasets are similar, the last column is the same - will or not them buy a product. In the first dataset, users who ...
2
votes
3answers
69 views
In adversarial machine learning, how does an attacker have access to the test and training dataset in order to poison it?
In the field of adversarial machine learning, machine learning models are vulnerable to attacks both on the test and training data set. However, how does the attacker get access to these datasets? How ...
2
votes
0answers
88 views
Is there a dataset for the detection of bomb explosions?
I would like to train a deep neural network to recognize bomb explosions. I was wondering if there is an open visual dataset for bomb explosion? Alternatively, if you know a good deep architecture or ...
0
votes
0answers
52 views
creating your own dataset similar to cityscapes format
I'm trying to train a neural network with my own dataset. The neural network can accept the cityscape format.
Is there any application that can give mask/segmented image, instance image, label IDs ...
2
votes
1answer
50 views
Do models train better if the dataset is more specific? (Semantic Segmentation / Bounding Box / Image classification)
I'm working on a project where there is a limited dataset of videos (about 200). We want to train a model that can detect a single class in the videos. That class can be of multiple different types ...
1
vote
2answers
89 views
TensorFlow 2.0 - Normalizing input to DNN (on structured data)
I have a structured dataset of around 100 gigs, and I am using DNN for classification in TF 2.0. Because of this huge dataset, I cannot load entire data in memory for training. So, I'll be reading ...
