Questions tagged [datasets]

For questions related to sets of data and their use in AI.

Filter by
Sorted by
Tagged with
1
vote
1answer
44 views

What data formats/pipelining are best to store and wrangle data which contains both text and float vectors?

Often in NLP project the data points contain both text and float embeddings, and it's very tricky to deal with. CSVs take up a ton of memory and are slow to load. But most the other data formats seem ...
1
vote
0answers
22 views

Size of dataset for feature vector with rare event

I'm doing a model to detect duplicate in my database (there is a lot of features that can be different but mean same object in the end) So I have my feature vector for my duplicate dataset which ...
1
vote
1answer
59 views

Do we train a logistic regression model using a dataset that is 3 times bigger than the validation dataset?

Suppose we have a data set $X$ that is split as $X_{\text{train}}$, $X_{\text{val}}$ and $X_{\text{test}}$ and the outcome variable is binary. Let's say we train three different models (logistic ...
1
vote
1answer
58 views

Is there an LSTM-based unsupervised learning algorithm to label a dataset of curves?

I have a big amount of light curves (image below). I am trying to label the points as signal or background (the signal appears usually periodically, several times, for a given light curve). More ...
2
votes
0answers
477 views

Deep Learning for radio signal classification with DeepSig dataset

I want to see if I can make my Software Defined Radio, SDR, to classify unknown radio signals with the help of an artificial neural network. That is, my SDR outputs a sequence of complex numbers (IQ-...
6
votes
1answer
3k views

How to detect LEGO bricks by using a deep learning approach?

In my thesis I dealt with the question how a computer can recognize LEGO bricks. With multiple object detection, I chose a deep learning approach. I also looked at an existing training set of LEGO ...
2
votes
0answers
33 views

What methods are there to generate artificial training examples based on existing training examples?

I have a small dataset (117 training examples) and many features (4005). Each of the training examples is binary labeled (healthy / diseased). Each feature represents the connectivity between two ...
0
votes
0answers
19 views

Consecutive frames can be discarded when training an SSD/YOLO?

Let's say I have a number of videos, and I want to train an SSD/YOLO (or FRCNN) to detect objects. In the case of a large amount of videos, there will be a lot of frames extracted and transferred to ...
0
votes
0answers
40 views

Is there a RNN that can predict the next substitute in a floorball match?

Floorball is a type of floor hockey. During the game, substitutions can be made. The team is also allowed to change players any time in the game; usually, they change the whole team. Individual ...
1
vote
1answer
43 views

transform Location data into int which will be used as input to ML model

I have features like state, city and location. Currently i am inserting this in respective tables in DB and tranform it using its primary key. eg. country,state,city = IN,Maharastra,Pune = 2,5,10 (...
13
votes
3answers
8k views

What is non-Euclidean data?

What is non-Euclidean data? Here are some sub-questions Where does this type of data arise? I have come across this term in the context of geometric deep learning and graph neural networks. ...
1
vote
2answers
32 views

What features should a dataset to predict monthly retail sales for a motorcycle spare parts shop have?

I am making an AI model to predict monthly retail sales of a motor cycle spare parts shop, for that to be possible I have to first create a dataset. The problem I am facing is what features should the ...
1
vote
0answers
35 views

identifying pattern in datasets

i am new to machine learning. i'm trying to identify driving pattern through accelerometer and gyroscope sensor. i have been collecting the data of both the sensors and have been storing them in .csv ...
0
votes
1answer
228 views

Parameters to calculate affluence in localities of Metro city

I have to calculate the affluence in localities of Metro city. To calculate affluence, I am considering a parameter per capita income. Where I can get a dataset of it? What are other parameters I ...
2
votes
1answer
215 views

Is a dataset of roughly 700 sentences of an average length of 15 words enough for text classification?

I'm building a customer assistant chatbot in Python. So, I am modelling this problem as a text classification task. I have available more or less 7 hundred sentences of an average length of 15 words (...
3
votes
0answers
58 views

How to train CNN such it eliminate dependent features and focuses on independent ones?

How we should train a CNN model when training dataset contains only limited number of cases, and the trained model is supposed to predict class (label) for several other cases, which has not seen ...
4
votes
2answers
1k views

Using GAN's to generate dataset for CNN training

I'm doing bachaleor thesis on traffic sign detection using single shot detector called YOLO. These single shot detectors can perform detection of objects in image and so they have specific way of ...
1
vote
0answers
37 views

how to estimate which item is bought most using AI

Problem: Maintaining the products is a big task for a retailer. If we can estimate using AI to predict which products will sell the most, we can maintain sufficient stocks of the product without ...
2
votes
1answer
132 views

Why isn't the Credit Card Fraud Detection dataset from Kaggle already balanced?

I am working on a credit card fraud detection problem using autoencoders. I have a question regarding the dataset I'll be using. I've downloaded the dataset for the above problem from Kaggle, which ...
0
votes
1answer
2k views

When training an object detection network for one class, should I include empty images in the dataset?

I fine tuned MobileNetSSD for object detection using a dataset with just one class (~4000 images). All the training images include at least one bounding box related to that class (no empty images). By ...
1
vote
0answers
34 views

How to preprocess a modified dataset so that a fitted CNN makes correct predictions on an un-modified version of the dataset?

for a school project I have been given a dataset containing images of plants and weeds. The goal is to detect when there is a weed in the pictures. The training and validation sets have already been ...
2
votes
0answers
42 views

Searching for dataset in specifix context for NLP, sentiment analysis

I would like to create an NPC engine for games that are in a fantasy context like Lord of the Rings, Warcraft, Skyrim, Dragon Age, etc. This engine should be able to determine the polarity of the ...
3
votes
1answer
207 views

Best way to create an image dataset for CNN

I am creating a dataset made of many images which are created by preprocessing a long time series. Each image is an array of (128,128) and the there are four classes. I would like to build a dataset ...
1
vote
0answers
23 views

input annotations quality check for large scale image data

while dealing with image data at very large scale, there are different sources where data is coming from. Often, we do not have any control over quality of labels/ annotations. I already do use ...
2
votes
0answers
35 views

People Counting Video Analytics: data acquisition parameters

I am interested in understanding how to choose data-acquisition parameters for the subject matter: Frame Resolution Frame rates (FPS) The goal is to have 'enough' (preferably the minimal) resolution ...
2
votes
0answers
41 views

Preset Intent list

I'm trying to build a chat analysis that could identify the intent of check price. Has any kind of preset intent list to train my chatbot already been done ? Data: A data set for "check price" ...
1
vote
1answer
234 views

Is 1mb an acceptable memory size for images being trained in a CNN?

I am using Tensorflow CNN to build an image classification/prediction model. Currently all the images in the dataset are each about 1mb in size. Most examples out there use very small images. The ...
3
votes
2answers
600 views

Which evaluation methods can I use for image segmentation?

I implemented an image segmentation pipeline and I trained it on the DICOM dataset. I compared the results of the model with manual segmentation to find the accuracy. Is there other methods for ...
7
votes
4answers
7k views

Traffic signs dataset [closed]

I'm looking for annotated dataset of traffic signs. I was able to find Belgium, German and many more traffic signs datasets. The only problem is these datasets contain only cropped images, like this: ...
5
votes
3answers
131 views

Is there a Python API for manipulating publicly available datasets? [closed]

In my daily machine learning / deep learning workflow, I often want to interact with a dataset in my code. Specifically, I would like to be able to load some module/package which can make sure that I ...
1
vote
1answer
69 views

Why would giving my AI more data make it perform worse?

So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample. Next I tried to get it to generate limericks using these 100,000 limericks. It generated ...
3
votes
2answers
300 views

Is a binary attribute type the same as binomial attribute type?

I am not sure if I can use the words binomial and binary and boolean as synonyms to describe a data attribute of a data set which has two values (yes or no). Are there any differences in the meaning ...
3
votes
1answer
227 views

Use cross-validation to train after model selection

I have been recently reading about model selection algorithms (for example to decide which value of the regularisation parameter or what size of a neural network to use, broadly hyper-parameters). ...
5
votes
1answer
881 views

What happens to the training data after your machine learning model has been trained?

I am completely new to all this, for the life of me I can't find the answer to this question anywhere on Google. What happens after you have used machine learning to train your model? What happens to ...
1
vote
1answer
23 views

Which features of a data set can be used for market campaigning using propensity scores?

A dataset contains so many fields in which there is both relevant and irrelevant field. If we want to do a market campaigning using propensity scoring, which fields of the data set are relevant? How ...
2
votes
1answer
103 views

Would YOLO be able to detect objects in “different” positions?

I have the following question about You Only Look Once (YOLO) algorithm, for object detection. I have to develop a neural network to recognize web components in web applications - for example, login ...
4
votes
1answer
272 views

Algorithm For Making Balanced Weapons In A Game?

I am trying to make balanced weapon pairs. So there are five stats per weapon, and I am simulating a number of combats (1000) with different stats randomized, and counting the win, lose, and draw of ...
5
votes
1answer
75 views

Is there a database somewhere of common lists? [closed]

I'm looking for a database or some machine readable document that contains common ordered lists or common short sets. e.g: ...
6
votes
2answers
153 views

How does rotating an image and adding new 'rotated classes' prevent overfitting?

From Meta-Learning with Memory-Augmented Neural Networks in section 4.1: To reduce the risk of overfitting, we performed data augmentation by randomly translating and rotating character images. ...
5
votes
2answers
348 views

How can I train a neural network for image classification when the dataset is small?

I need to train a convolutional neural network to classify snake images. The problem is that I have only a small number of images available for some snake types. So, what is the best approach to train ...
5
votes
1answer
258 views

For each epoch, can I use only on a subset of the full training dataset to train the neural network?

If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini-batch gradient descent on this entire dataset is too high, can I ...
3
votes
1answer
55 views

Automatic image classification

I'm a complete newbie to NNs, and I need your advice. I have a set of images of symbols, and my goal is to categorize and divide them into groups of symbols that look alike. Without teaching NN ...
1
vote
2answers
313 views

Normalization for well known data sets like coco-text and total text data set

In the data-sets like coco-text and total-text, the images are of different sizes (height*width). I'm using these data sets for text detection. I want to create a DNN model for this. So input data ...
3
votes
1answer
98 views

Is there a way to predict points on a map?

I have a data set with historical information of some events (let's say event A and event B),these events describe the discovery of land mines, the coordinates of the event and the date of the event; ...
0
votes
1answer
396 views

Are standard deviation, variance, skew good features for ML?

Pretty simple question here: Is it useful to use the standard deviation, skew, kurtosis, or any other extrapolatory stats as features, and if so in which problem sets? In this case, I am talking ...
1
vote
1answer
67 views

Can we code rules for an agent in python language other than predicate calculus?

I have a medical dataset with 14000 rows dataset with 900 attributes. I have to predict disease severity using that. I would like to know whether we can write rules in python language for training an ...
1
vote
0answers
47 views

Data Interpretation technique

In the model generation, in machine learning (consider supervised) If some data change the previous model function drastically then we should study that data. Does it happen? How to handle such ...
24
votes
5answers
27k views

How can I deal with images of variable dimensions when doing image segmentation?

I'm facing the problem of having images of different dimensions as inputs in a segmentation task. Note that the images do not even have the same aspect ratio. One common approach that I found in ...
4
votes
1answer
2k views

What are some tools for labelling images and videos? [closed]

I am looking for AI tools for image and video labeling. It would be great if someone could share its own experience as well suggest tools. I also have some questions Which tools are available? (...
1
vote
0answers
215 views

Feature set out of grayscale Images for training a neural network?

Previously I had trained a Neural Networkupon 20,000 character images. This Neural Net generally works well, it uses ...