Questions tagged [datasets]

For questions related to sets of data and their use in AI.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Predicting single floats based on set of 2 feature arrays each of 100 values

I am trying to predict audio to video desynchronization based on ser od two arrays of lenght 100 which consist of coresponding audio and video samples. The problem is that my labels are single floats (...
1
vote
0answers
17 views

What are the types of data in which the order of instances does matter?

In general, the order of instances in the datasets that are used in machine learning is immaterial. But there are exceptions. Timeseries data is one such exception I know. Consider the following two ...
0
votes
0answers
8 views

What to predict in a limited transaction dataset?

I have been given a task with a real transaction dataset. The task is to predict something using either logistic regression or simple binary classification. The columns are as follow: Transaction ID ...
26
votes
5answers
31k views

How can I deal with images of variable dimensions when doing image segmentation?

I'm facing the problem of having images of different dimensions as inputs in a segmentation task. Note that the images do not even have the same aspect ratio. One common approach that I found in ...
0
votes
0answers
15 views

What are the pros and cons of using a normal positional encoding in an adjacency matrix?

I understand that a normal positional encoding helps a transformer to understand pictures better and that it allows the (otherwise permutational invariant transformer-network) to create relationships ...
1
vote
1answer
56 views

How exactly does MICE imputation combine multiple datasets into one?

I'm trying to understand Multiple Imputation with Chained Equation (MICE) imputation process (a statistical method for imputing missing data). I have read some articles and I have understood how the ...
2
votes
0answers
25 views

Does the Frechet Inception Distance (FID) consider color?

I was wondering if the Frechet inception distance for two colored datasets would be the same than the FID calculated for the same datasets converted to grayscale. I know that it depends on the feature ...
0
votes
0answers
38 views

Predicting height of a person using one image [closed]

I need help with trying to find why this algorithm is not working I am trying to predict the height of a person from a single picture this will be able to help us predict the height without multiple ...
3
votes
1answer
71 views

How to perform binary classification when one class is more predominant than the other?

Assuming we have big $m \times n$ input dataset, with $m \times 1$ output vector. It's a classification problem with only two possible values: either $1$ or $0$. Now, the problem is that almost all ...
3
votes
0answers
18 views

Do any practical deep learning algorithms deal with tensors containing non-real entries?

In deep learning, most of the applications are from text and images. Both text and images can be converted into a tensor of real numbers. Other than both mentioned above, there may be some other real-...
2
votes
1answer
37 views

Is there any simple example for volumetric data except from physics and medicine?

Recently I heard about the term volumetric data. The definition for volumetric data is as follows #1: Definition Volumetric data is typically a set S of samples $(x, y, z, v)$, representing the value ...
1
vote
1answer
80 views

Is it possible to know the distance objects are from camera based on only knowing one object's height?

I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
4
votes
1answer
310 views

Which algorithm can I use to minimise the number of wins of 2 weapons that fight each other in a game?

I have a game that involves 2 weapons, which fight against each other. Each weapon has 5 features/statistics, which have certain range. I can simulate the game $N$ times with randomly initialised ...
1
vote
1answer
46 views

Generating a dataset from data with "assumed" lables

I've got a task similar to the following: Out of x amount of people, I need to predict, who could be a good athlete and who not. The thing is, I don't have data on the athletic performance of those ...
1
vote
0answers
54 views

What should we do when the new data drastically change the current model?

In machine learning (in particular, supervised learning), if some new data changes the previous model/function drastically, then I think we should study that data. Does it happen? How to handle such a ...
2
votes
0answers
43 views

What is currently the most competitive regression models/algorithms used on the "Boston Housing Prices" dataset?

I'm currently looking for a standard data-set to test some new regression models I've been developing on, and the "Boston Housing Prices" data-set seemed to stand out, since it's very ...
1
vote
1answer
239 views

Parallelised my training or the dataset

I have some plans in working with Reinforcement Learning in order to predict the stock price movement. For a stock like TSLA, some training features might be the pivot price values and the set of the ...
2
votes
1answer
29 views

Can I use the transformers for the prediction of historical data?

Can I use the transformers for the prediction of wind power with the historical data? Dataset Datetime, Ambient temperature (Degree), Dewpoint (Degree), Relative Humidity\n (%), Air Pressure, Wind ...
8
votes
2answers
346 views

What are examples of techniques to prevent bias in artificial intelligence systems?

I recently read an article about how artificial intelligence replicates human stereotypes when applied to biased datasets. What are examples of techniques to prevent bias (and stereotypes) in ...
4
votes
1answer
69 views

How to determine the quality of synthetic data?

I'm working on a VAE model to produce synthetic data of X-Ray diffraction spectrums. I try to figure out how I can measure the quality of the spectrums. The goal would be to produce synthetic data ...
1
vote
1answer
68 views

Is there a way of automatically drawing bounding boxes around interested objects?

Given thousands of images, where some of the images contain target objects and others do not, is there an easy way of drawing bounding boxes on these target objects rather than relying on manual ...
3
votes
2answers
204 views

How to fix time dimension in time varying data-sets using deep learning model for classification?

Dataset Description I am working on famous ABIDE Autism Datasets. The dataset is very big in a sense that it has more than 1000 subjects containing half of them as autisitic and other half as ...
2
votes
1answer
57 views

Is it compulsary to normalize the dataset if doing so can negatively impact a Binary Logistic regression performance?

I am using raw data set with 4 feature variables (Total Cholesterol, Systolic Blood Pressure, Diastolic Blood Pressure, and Cigraeette count) to do a Binominal Classification (find stroke likelihood) ...
1
vote
0answers
26 views

Recommended way to spilt image sequence for training/validation/testing

For object detection tasks I have a few minutes of video footage from a surveillance camera, converted to a sequence of images and ground truth bounding boxes for all people walking by. Now what's the ...
2
votes
1answer
33 views

How do you handle unbalanced image datasets?

I have an image data set on which I am training a CNN. The data set is slightly unbalanced. So, my solution up till now was to delete some images of the majority class. But I now realize that there ...
9
votes
3answers
747 views

Is it okay to use publicly available Instagram videos to train an AI?

Since I haven't found any good training data for my university project, I want to use pictures and videos from public Instagram profiles. Am I allowed to do that?
2
votes
1answer
137 views

Would YOLO be able to detect objects in "different" positions?

I have the following question about You Only Look Once (YOLO) algorithm, for object detection. I have to develop a neural network to recognize web components in web applications - for example, login ...
1
vote
1answer
61 views

Do we train a logistic regression model using a dataset that is 3 times bigger than the validation dataset?

Suppose we have a data set $X$ that is split as $X_{\text{train}}$, $X_{\text{val}}$ and $X_{\text{test}}$ and the outcome variable is binary. Let's say we train three different models (logistic ...
1
vote
0answers
46 views

Which of the following probability distribution is generating an iid dataset?

Let $X_1, X_2$ be two discrete random variables. Each random variable takes two values: $1, 2$ The probability distribution $p_1$ over $X_1, X_2$ is given by $$p_1(X_1=1, X_2 = 1) = \dfrac{1}{4}$$ $$...
0
votes
0answers
25 views

Modelling of output neuron for mixed features?

Dataset in artificial intelligence, in general, consists of some features (say $n$). Assume that $m$ among them are output features. I want to model this function using neural network. So, input to my ...
1
vote
1answer
44 views

How can we "draw i.i.d" from any probability distribution?

Consider the following paragraph from 2 Learning in High Dimensions in from of the paper titled Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges ...
2
votes
1answer
167 views

How are weights for weighted x-entropy loss on imbalanced data calculated?

I am trying to build a classifier which should be trained with the cross entropy loss. The training data is highly class-imbalanced. To tackle this, I've gone through the advice of the tensorflow docs ...
0
votes
1answer
42 views

Why disentangling the features of variation in representation?

Consider the following excerpt from abstract of the research paper titled Better Mixing via Deep Representations by Yoshua Bengio et al. It has been hypothesized, ...
0
votes
0answers
36 views

What are mathematically the factors of variation in deep learning?

The following paragraph from an answer tells us about factors of variation Factors of variation are some factors which determine varieties in observed data. If that factors change, the behaviour of ...
2
votes
5answers
84 views

How do I increase the size of an (almost) balanced dataset?

I am trying to add more data points in my (almost) balanced dataset for training my neural network. I have come across techniques such as SMOTE or Random Over Sampling, but they work best for ...
0
votes
1answer
37 views

How to handle an unbalanced dataset when training object detection algorithms?

I am training an object detection model, and I have some very highly unbalanced data annotations. I have almost 11,000 images, all with dimensions of 1024 $\times$ 1024. Within those images I have the ...
1
vote
0answers
38 views

What's the best way to feed stories to a neural network?

I'm trying to train a model that would generate stories. I have a dataset of 2000 stories prepared. They are tokenized and one-hot encoded. I can't load them all at once as a one big dataset, because ...
0
votes
2answers
39 views

Why data required for hyperparameter tuning is considered as an additional data?

Any parametric model may have parameters as well as hyperparameters. Learning algorithm deals with parameters and hyperparameters should be dealt outside learning algorithm. Consider the following ...
1
vote
0answers
33 views

Is it true that real world data is highly discontinuous?

A function $f$ is said to be continuous at a point $c$ if it satisfies three properties: Should be defined at the point $c$ Left and right-hand limits at $c$ must be equal i.e., the limit must exist ...
0
votes
0answers
22 views

Adding data to training results in loss random peaks

I have succesfully trained ssd_mobilenet_v2_keras for object detection, with a dataset of about 3700 images. Now I have more images to add. I tried adding only a few images (150-300) to see what ...
0
votes
0answers
21 views

Which Neural Network Topology to choose, are Transformers suitable?

I have a regression problem and I am not quite sure which architecture to choose. I never worked with transformers before, but I generally understand how they work and I think they might be suitable. ...
0
votes
1answer
21 views

Data analysis before feeding to ML pipeline

I'm new to machine learning and I've been working through a dataset of ~3000 records with ~100 features. I've been hand rolling Python and R scripts to analyse the data. For example, plotting the ...
0
votes
0answers
13 views

Weight for Samples on SVM (Support Vector Machine)

There is a option sample_weight in fit(X[, y, sample_weight]) function (OneClassSVM, sklearn library). If I use the option ...
1
vote
0answers
22 views

Training on the dataset in parts vs training on the whole dataset

What is the difference between these two situations? are they the same ? #1 : train a model 20 epochs on the whole dataset #2 : divide dataset into n-parts then train the model 20 epochs on each part ...
1
vote
0answers
46 views

Batch normalization for multiple datasets?

I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data. I was told that batch ...
0
votes
1answer
27 views

An online editor that allows data labeling format [closed]

I have a set of students (~20) that will work on annotating data for an NLP project. The annotation task will be as in the following: ...
1
vote
2answers
47 views

Train Validation Test Splitting After or Before Data Augmentation?

I have seen tutorials online saying that you should do data augmentation AFTER doing the train/val/test split. However, when I go online to read some research papers, I see numerous instances of ...
1
vote
0answers
15 views

Comparing results of different image splicing methods on a part of the CASIA 2.0 dataset

So I am working on an image splicing detection algorithm using ResNet-50 model. I am using the CASIA 2.0 dataset which consists of 7491 Authentic images and 5123 Fake images. However out of the fake ...
1
vote
1answer
87 views

What are the labels of EMNIST? [closed]

I'm using pytorch and downloading the EMNIST dataset using ...

1
2 3 4 5