Questions tagged [datasets]

For questions related to sets of data and their use in AI.

Filter by
Sorted by
Tagged with
24
votes
5answers
27k views

How can I deal with images of variable dimensions when doing image segmentation?

I'm facing the problem of having images of different dimensions as inputs in a segmentation task. Note that the images do not even have the same aspect ratio. One common approach that I found in ...
13
votes
3answers
8k views

What is non-Euclidean data?

What is non-Euclidean data? Here are some sub-questions Where does this type of data arise? I have come across this term in the context of geometric deep learning and graph neural networks. ...
11
votes
4answers
236 views

How do I select the relevant features of the data?

Recently I was working on a problem to do some cost analysis of my expenditure for some particular resource. I usually make some manual decisions from the analysis and plan accordingly. I have a big ...
8
votes
2answers
325 views

What are examples of techniques to prevent bias in artificial intelligence systems?

I recently read an article about how artificial intelligence replicates human stereotypes when applied to biased datasets. What are examples of techniques to prevent bias (and stereotypes) in ...
7
votes
4answers
7k views

Traffic signs dataset [closed]

I'm looking for annotated dataset of traffic signs. I was able to find Belgium, German and many more traffic signs datasets. The only problem is these datasets contain only cropped images, like this: ...
6
votes
4answers
730 views

What are some datasets to train an MLP on simple tasks? [closed]

I have implemented an MLP. Now, I want to train it to solve simple tasks. Are there any data sets to train the MLP on simple tasks, that is, tasks with a small number of inputs and outputs? I ...
6
votes
3answers
284 views

Has anybody tried unsupervised deep learning from youtube videos?

YouTube has a huge amount of videos, many of which also containing various spoken languages. This would presumably provide something like the data that a "challenged" baby would experience - "...
6
votes
1answer
3k views

How to detect LEGO bricks by using a deep learning approach?

In my thesis I dealt with the question how a computer can recognize LEGO bricks. With multiple object detection, I chose a deep learning approach. I also looked at an existing training set of LEGO ...
6
votes
2answers
151 views

How does rotating an image and adding new 'rotated classes' prevent overfitting?

From Meta-Learning with Memory-Augmented Neural Networks in section 4.1: To reduce the risk of overfitting, we performed data augmentation by randomly translating and rotating character images. ...
5
votes
2answers
348 views

How can I train a neural network for image classification when the dataset is small?

I need to train a convolutional neural network to classify snake images. The problem is that I have only a small number of images available for some snake types. So, what is the best approach to train ...
5
votes
3answers
131 views

Is there a Python API for manipulating publicly available datasets? [closed]

In my daily machine learning / deep learning workflow, I often want to interact with a dataset in my code. Specifically, I would like to be able to load some module/package which can make sure that I ...
5
votes
1answer
880 views

What happens to the training data after your machine learning model has been trained?

I am completely new to all this, for the life of me I can't find the answer to this question anywhere on Google. What happens after you have used machine learning to train your model? What happens to ...
5
votes
2answers
621 views

What is the effect of mislabeled training data?

Collecting and labeling training data for supervised learning tasks is incredibly time-consuming and costly. For instance, let's say you wrote a script that went on Google images and got you 5000 ...
5
votes
2answers
2k views

Does the quality of training images affect the accuracy of the neural network?

I just got into AI few months ago. I noticed most of the images in training datasets are usually low quality( almost pixelated). Does the quality of training images affect the accuracy of the neural ...
5
votes
1answer
106 views

How can one find / collect data for, and come up with ideas for, using Deep Learning / AI to improve one's everyday life?

I can see a lot of tutorials and examples about using TensorFlow and other free, open-source AI/ML/DL frameworks on enterprise level where enough data was collected for such AI solutions. How can one ...
5
votes
1answer
75 views

Is there a database somewhere of common lists? [closed]

I'm looking for a database or some machine readable document that contains common ordered lists or common short sets. e.g: ...
5
votes
1answer
258 views

For each epoch, can I use only on a subset of the full training dataset to train the neural network?

If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini-batch gradient descent on this entire dataset is too high, can I ...
5
votes
1answer
103 views

Find anomalies from records of categorical data

I have a data-set with $m$ observations and $p$ categorical variables (nominal), each variable $X_1, X_2,\dots, X_p$ has several different possible values. Ultimately, I am looking for a way to find ...
4
votes
4answers
143 views

How to classify language as friendly or aggressive with AI?

Just for the purpose of learning I'd like to classify the likeliness of a tweet being in aggressive language or not. I was wondering how to approach the problem. I guess I need first train my neural ...
4
votes
2answers
150 views

Why do we need both the validation set and test set?

I know that this has been asked a hundred times before, however, I was not able to find a question (and an answer) which actually answered what I wanted to know, respectively, which explained it in a ...
4
votes
3answers
191 views

Would this relatively small dataset be enough to train a CNN?

Scenario: I am trying to create a dataset with images of choice for different animal classes. I am going to train those images for classification using CNN. Problem: Let's assume I somehow don't have ...
4
votes
1answer
132 views

How should I generate datasets for a SARSA agent when the environment is not simple?

I am currently working on my master's thesis and going to apply Deep-SARSA as my DRL algorithm. The problem is that there is no datasets available and I guess that I should generate them somehow. ...
4
votes
1answer
61 views

How could decision tree learning algorithms cope with imbalanced classes?

Decision trees and random forests may or not be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article Using Random Forest to ...
4
votes
2answers
1k views

Using GAN's to generate dataset for CNN training

I'm doing bachaleor thesis on traffic sign detection using single shot detector called YOLO. These single shot detectors can perform detection of objects in image and so they have specific way of ...
4
votes
1answer
2k views

What are some tools for labelling images and videos? [closed]

I am looking for AI tools for image and video labeling. It would be great if someone could share its own experience as well suggest tools. I also have some questions Which tools are available? (...
4
votes
1answer
121 views

What are some concrete steps to deal with the vanishing gradient problem?

I am training an ANN for classification between 3 classes. The ANN has an input layer, one hidden layer and a 3 node output layer. The problem I am facing is that the output being produced by the 3 ...
4
votes
1answer
82 views

Can we use ML to do anything else other than predicting (in the case of mathematical problems)?

(The math problem here just serves as an example, my question is on this type of problems in general). Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
4
votes
1answer
271 views

Algorithm For Making Balanced Weapons In A Game?

I am trying to make balanced weapon pairs. So there are five stats per weapon, and I am simulating a number of combats (1000) with different stats randomized, and counting the win, lose, and draw of ...
4
votes
1answer
416 views

Is there any free dataset of source code along with natural language description? [closed]

I am currently working on my MSc thesis, where I try to automatically convert description of an algorithm in natural language to source code in Python (the algorithms are quite simple, like revert an ...
4
votes
1answer
101 views

How to fill missing values in a dataset where some properties can be inputs and outputs?

I have a dataset with missing values, I would like to use machine learning methods to fill. In more detail, there are $n$ individuals, for which up to 10 properties are provided, all numerical. The ...
4
votes
2answers
106 views

Is a basic neural network architecture better with small datasets?

I'm currently trying to predict 1 output value with 52 input values. The problem is that I only have around 100 rows of data that I can use. Will I get more accurate results when I use a small ...
4
votes
3answers
65 views

Can machine learning help me digest asymmetrical order descriptions?

I have order data, here's a sample: ...
4
votes
1answer
37 views

Looking to build, compile, and/or find dataset for serial-parallelized code examples

I'm looking to perform two tasks: Train a classifier to classify code as serial or parallel Train a generative algorithm to generate parallel code from serial For the first task a simple scraper ...
4
votes
1answer
1k views

Dataset containing images of varying dimensions and orientations

I am new to deep learning. I have a dataset of images of varying dimensions of a certain object. A few images of the object are also in varying orientations. The objective is to learn the features ...
4
votes
0answers
2k views

How many training data required for GAN?

I'm beginning to study and implement GAN to generate more dataset. I'll just try to experiment with state-of-the-art GAN models as described in here https://paperswithcode.com/sota/image-generation-on-...
4
votes
0answers
109 views

Are there any easy ways to create annotated training images for object detection?

For the purposes of object detection, are there any easy ways to create annotated training images? For example, if we have $10,000$ images and want to draw bounding boxes on 2 objects for each image, ...
4
votes
0answers
96 views

Are there datasets to solve differential equations in a supervised fashion? [closed]

Are there datasets to solve differential equations in a supervised fashion? More precisely, the input is a differential equation and the label should be the general solution to that differential ...
4
votes
2answers
85 views

How should I select the features for predicting diseases (in particular when patients specify their health issues)?

My aim is to train a model for predicting diseases. Now, according to this Wikipedia article, diseases are classified based on the following criteria in general: Causes (of the disease) Pathogenesis (...
3
votes
2answers
91 views

Are there tools to help labelling images? [closed]

I need to manually classify thousands of pictures into discrete categories, say, where each picture is to be tagged either A, B, or C. Edit: I want to do this work myself, not outsource / crowdsource ...
3
votes
2answers
120 views

Can we use genetic algorithms to evolve datasets?

Genetic algorithms are used to solve many optimization tasks. If I have a dataset, can I evolve it with a genetic algorithm to create an evolved version of the same dataset? We could consider each ...
3
votes
1answer
712 views

What are “proxy data sets” in machine learning?

The paper Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration uses the term "proxy data sets" in this way To develop DL ...
3
votes
1answer
105 views

Is it possible to classify resistors using ResNet50?

I want to train ResNet50 model using resistor images like below: I tried it by collecting data from google images and there were quite few. So accuracy was very low (around %10) but I wonder If it is ...
3
votes
6answers
74 views

How can I cluster this data frame with several features and observations?

How can I cluster the data frame below with several features and observations? And how would I go about determining the quality of those clusters? Is k-NN appropriate for this? ...
3
votes
1answer
72 views

Can you let specific data impact a neural network more than other data?

I have a lot of empty values in my dataset, so I want to let my neural network 'learn more' on the rows that have no empty values because these rows are of higher importance. Is there a way to do ...
3
votes
2answers
107 views

Is there a possibility that there is no relationship between some inputs and outputs?

I'm doing machine learning projects. I took a look at many datasets I worked with, mostly there are already famous datasets that everyone uses. Let's say I decided to make my own dataset. Is there a ...
3
votes
2answers
597 views

Which evaluation methods can I use for image segmentation?

I implemented an image segmentation pipeline and I trained it on the DICOM dataset. I compared the results of the model with manual segmentation to find the accuracy. Is there other methods for ...
3
votes
1answer
227 views

Use cross-validation to train after model selection

I have been recently reading about model selection algorithms (for example to decide which value of the regularisation parameter or what size of a neural network to use, broadly hyper-parameters). ...
3
votes
4answers
161 views

If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

Suppose one trains a CNN to determine if something was either a cat/dog or neither (2 classes), would it be a good idea to assign all cats and dogs to one class and everything else to another? Or ...
3
votes
2answers
361 views

How to generate a dataset for question answering from books (like Facebook's bAbI)?

I wanted to train a chatbot for answering questions from books. I am trying to use Dynamic Memory Networks to do so. How can I generate a data set, as Facebook did in the case of bAbI tasks, so that ...
3
votes
1answer
61 views

Why different images of the same person, under some restrictions, are in a 50 dimension manifold?

In this lecture (starting from 1:31:00) the professor says that the set of all images of a person lives in a low dimensional surface (compared the the set of all possible images). And he says that the ...

1
2 3 4 5