Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [datasets]

For questions based on proper usage of data for Machine Learning ad other Algorithms to extract useful information.

0
votes
0answers
6 views

input annotations quality check for large scale image data

while dealing with image data at very large scale, there are different sources where data is coming from. Often, we do not have any control over quality of labels/ annotations. I already do use ...
1
vote
0answers
12 views

People Counting Video Analytics: data acquisition parameters

I am interested in understanding how to choose data-acquisition parameters for the subject matter: Frame Resolution Frame rates (FPS) The goal is to have 'enough' (preferably the minimal) resolution ...
1
vote
0answers
18 views

Preset Intent list

I'm trying to build a chat analysis that could identify the intent of check price. Has any kind of preset intent list to train my chatbot already been done ? Data: A data set for "check price" ...
0
votes
1answer
28 views

Is 1mb an acceptable memory size for images being trained in a CNN?

I am using Tensorflow CNN to build an image classification/prediction model. Currently all the images in the dataset are each about 1mb in size. Most examples out there use very small images. The ...
0
votes
1answer
11 views

how image segmentation evaluated?

I implemented the segmentation on my DICOM Dataset and it is compared with manual segmentation to find the accuracy of segmentation, is there other methods for evaluation?
2
votes
3answers
62 views

Traffic signs dataset

I'm looking for annotated dataset of traffic signs. I was able to find Belgium, German and many more traffic signs datasets. The only problem is these datasets contain only cropped images, like this: ...
3
votes
1answer
35 views

Python API for publicly available datasets

In my daily machine learning / deep learning workflow, I often want to interact with a dataset in my code. Specifically, I would like to be able to load some module / package which can make sure that ...
2
votes
1answer
55 views

Why would giving my AI more data make it perform worse?

So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample. Next I tried to get it to generate limericks using these 100,000 limericks. It generated ...
0
votes
0answers
51 views

How can I deal with 4 channel image with pytorch?

I have 4 channel RGB-D data. (RGB 3-channel image with depth information per pixel) What I want is, use these data as input for trained network (and this one is not made by me.). However, this ...
3
votes
2answers
24 views

Is a binary attribute type the same as binomial attribute type?

I am not sure if I can use the words binomial and binary and boolean as synonyms to describe a data attribute of a data set which has two values (yes or no). Are there any differences in the meaning ...
3
votes
1answer
64 views

Use cross-validation to train after model selection

I have been recently reading about model selection algorithms (for example to decide which value of the regularisation parameter or what size of a neural network to use, broadly hyper-parameters). ...
5
votes
1answer
133 views

What happens to the training data after your machine learning model has been trained?

I am completely new to all this, for the life of me I can't find the answer to this question anywhere on Google. What happens after you have used machine learning to train your model? What happens to ...
1
vote
1answer
17 views

Which features of a data set can be used for market campaigning using propensity scores?

A dataset contains so many fields in which there is both relevant and irrelevant field. If we want to do a market campaigning using propensity scoring, which fields of the data set are relevant? How ...
1
vote
0answers
24 views

YOLO - How much is the position of the object relevant in learning?

I have the following question about You Only Look Once (YOLO) algorithm, for object recognition in CNNs. I have to develop a neural network to recognize web components in web applications - for ...
4
votes
1answer
84 views

Algorithm For Making Balanced Weapons In A Game?

I am trying to make balanced weapon pairs. So there are five stats per weapon, and I am simulating a number of combats (1000) with different stats randomized, and counting the win, lose, and draw of ...
5
votes
2answers
56 views

Is there a database somewhere of common lists?

I'm looking for a database or some machine readable document that contains common ordered lists or common short sets. e.g: ...
6
votes
2answers
72 views

How does rotating an image and adding new 'rotated classes' prevent overfitting?

From Meta-Learning with Memory-Augmented Neural Networks in section 4.1: To reduce the risk of overfitting, we performed data augmentation by randomly translating and rotating character images. ...
5
votes
2answers
90 views

Working with large datasets

If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini batch gradient descent on this entire dataset is too high, can I ...
2
votes
1answer
38 views

Automatic image classification

I'm a complete newbie to NNs, and I need your advice. I have a set of images of symbols, and my goal is to categorize and divide them into groups of symbols that look alike. Without teaching NN ...
1
vote
2answers
30 views

Normalization for well known data sets like coco-text and total text data set

In the data-sets like coco-text and total-text, the images are of different sizes (height*width). I'm using these data sets for text detection. I want to create a DNN model for this. So input data ...
3
votes
1answer
66 views

Is there a way to predict points on a map?

I have a data set with historical information of some events (let's say event A and event B),these events describe the discovery of land mines, the coordinates of the event and the date of the event; ...
-1
votes
1answer
96 views

Are standard deviation, variance, skew good features for ML?

Pretty simple question here: Is it useful to use the standard deviation, skew, kurtosis, or any other extrapolatory stats as features, and if so in which problem sets? In this case, I am talking ...
0
votes
1answer
39 views

Can we code rules for an agent in python language other than predicate calculus?

I have a medical dataset with 14000 rows dataset with 900 attributes. I have to predict disease severity using that. I would like to know whether we can write rules in python language for training an ...
1
vote
0answers
28 views

Data Interpretation technique

In the model generation, in machine learning (consider supervised) If some data change the previous model function drastically then we should study that data. Does it happen? How to handle such ...
3
votes
2answers
712 views

Convolutional neural networks with input images of different dimensions - Image segmentation

I'm facing the problem of having images of different dimensions as inputs in a segmentation task. Note that the images do not even have the same aspect ratio. One common approach that I found in ...
1
vote
0answers
17 views

Feature set out of grayscale Images for training a neural network?

Previously I had trained a Neural Networkupon 20,000 character images. This Neural Net generally works well, it uses ...
4
votes
3answers
46 views

Can machine learning help me digest asymmetrical order descriptions?

I have order data, here's a sample: ...
0
votes
0answers
15 views

Dataset of UIs augmented by Machine learning

Does anyone know of any dataset repository containing user interfaces augmented in some way by machine learning? Ideally something like Rico, but listing ui components that uses machine learning ...
0
votes
0answers
24 views

Retraining a trained model in real-time

I have a trained CNN model which is 95% Accurate. It classifies, a family and a group of friends who are eating in a restaurant. I made two folders inside the Training and Testing set with the images ...
2
votes
0answers
36 views

Shifting training data

I want to create a neural network and train it on some data, however I want to be able to create a new model without retraining it from the start. An example, I have 1000 data points in my training ...
6
votes
3answers
224 views

Has anybody tried unsupervised deep learning from youtube videos?

YouTube has a huge amount of videos, many of which also containing various spoken languages. This would presumably provide something like the data that a "challenged" baby would experience - "...
4
votes
1answer
27 views

Looking to build, compile, and/or find dataset for serial-parallelized code examples

I'm looking to perform two tasks: Train a classifier to classify code as serial or parallel Train a generative algorithm to generate parallel code from serial For the first task a simple scraper ...
4
votes
3answers
81 views

Good idea to assign different objects to same class?

Suppose one trains a CNN to determine if something was either a cat/dog or neither (2 classes), would it be a good idea to assign all cats and dogs to one class and everything else to another? Or ...
0
votes
1answer
58 views

How to make training data for AI

Right now I'm planning to make a DNN for classifying taste of crystals with their molecular structure which include information like no of atoms mass of each atoms atomic no of atoms. How should I ...
2
votes
0answers
49 views

Classification Learning - Normalization of time series and live usage

UPDATE: The tables look messed up so i put them on pastebin for better visibility. https://pastebin.com/gDX28uVF I am using Neural Network with different learning types (for example Standard ...
0
votes
2answers
40 views

A way to give more weight to particular data?

Let us for these purposes say with are working with any feed forward neural network. Let us also say, that we know beforehand that certain portion of our dataset arsignificantly more impactful or ...
3
votes
1answer
63 views

Find anomalies from records of categorical data

I have a data-set with m observations and p categorical variables (nominal), each variable ...
0
votes
0answers
242 views

PASCAL VOC dataset

I am trying to train a deep network on PASCAL VOC 2012 dataset. But the number of images are a bit too few. There are some papers that mention about augmented PASCAL VOC set (like PSPNet, Section 5.3)....
5
votes
1answer
83 views

What are some concrete steps to deal with the vanishing gradient problem?

So I am training an ANN for classification between 3 classes. The ANN has an input layer, one hidden layer and a 3 node output layer. The problem I am facing is that the output being produced by the ...
1
vote
2answers
152 views

Data-set values feature scaling: sigmoid vs tanh

As many papers point out, for better learning curve of a NN, it is better for a data-set to be normalized in a way such that values match a Gaussian curve. Does this process of feature normalization ...
1
vote
1answer
282 views

Human Height estimation using person detection techniques

I have a pedestrian dataset and would like to estimate human height in a video survillance using person detection techniques like YOLO Darknet or SSD (Single Shot Detectors). Would this technique work?...
1
vote
1answer
301 views

How to label edited images after data augmentation

I am new to neural networks, I've only started studying and learning about the subject a year ago, and I just started building my first neural network. The project is a little bit ambitious: A ...
1
vote
2answers
70 views

Machine learning and machine generated content conflict problem

Machine learning and NN trainings as a part of ML is based on data that was gotten from real world and inserted into virtual space by humans. Meanwhile NN are also used for data generation. Each year ...
2
votes
1answer
119 views

What is the effect of mislabeled training data?

Collecting and labeling training data for a supervised learning tasks is incredibly time-consuming and costly. For instance; Let's say you wrote a script that went on Google images and got you 5000 ...
1
vote
0answers
50 views

Orientation of data set before training simple ANN's

Well, I am new to implementing ANN's and there is something that i want to know. It maybe a bit silly though. I just wanted to know that if we have a simple data set say dependent only on a single ...
0
votes
1answer
25 views

Recommendation system based on content type

I am new to this field and would like to know that for what kind of data types other than images, recommendation system can be created using machine learning. Suppose for contents like audio or video, ...
3
votes
1answer
244 views

Is there any free dataset of source code along with natural language description?

I am currently working on my MSc thesis, where I try to automatically convert description of an algorithm in natural language to source code in Python (the algorithms are quite simple, like revert an ...
2
votes
0answers
112 views

Preprocessing of training dataset for machine learning

I'm developing a log analyzer to predict and find errors in an equipament. Each logged data contains the following format: ...
4
votes
1answer
99 views

How can one find / collect data for, and come up with ideas for, using Deep Learning / AI to improve one's everyday life?

I can see a lot of tutorials and examples about using TensorFlow and other free, open-source AI/ML/DL frameworks on enterprise level where enough data was collected for such AI solutions. How can one ...
3
votes
1answer
256 views

Dataset containing images of varying dimensions and orientations

I am new to deep learning. I have a dataset of images of varying dimensions of a certain object. A few images of the object are also in varying orientations. The objective is to learn the features ...