Questions tagged [imbalanced-datasets]

For questions that involve imbalanced (or unbalanced) datasets.

Filter by
Sorted by
Tagged with
0 votes
0 answers
17 views

Is it necessary that the number of samples of one class be balanced with other classes in a classification problem?

Consider a classification problem using machine learning techniques (e.g. malware detection). In such a problem, is it necessary that the number of samples from each class (in the mentioned example, ...
user16385455's user avatar
0 votes
1 answer
57 views

Why would balancing be so helpful when the imbalance is minimal?

I have a binary classification problem with a modest-to-none class imbalance (33% positive class-66% negative class). When I don't impose class balance, my XGBoost model produces no positive class ...
bonzo_pippinpaddle's user avatar
0 votes
0 answers
15 views

Why my deep learning model (FCNN/ 1DCNN) fails to learn when training on medical dataset?

I am working on a project to predict the severity of the disease, Hemophilia using a deep learning model(FCNN or 1DCNN). I am working based on the information provided in this article: https://www....
54rnd's user avatar
  • 1
0 votes
1 answer
56 views

How to evaluate binary classifier on imbalanced dataset?

I have trained a Decision Tree model on an imbalanced dataset. I got the following results for the test set from the sklearn and imblearn classification reports (attached below). Moreover, the other ...
Zal's user avatar
  • 7
1 vote
1 answer
45 views

Fine tuning a Deep Learning model post training

I have trained a CNN in a binary classification problem, however the original problem has 6 different classes, of which, I am only interested in classifying one, so if it is that certain class or not....
NeuroEng's user avatar
  • 121
0 votes
1 answer
151 views

How to interpret binary classification metrics on an imbalanced data set?

I have an imbalanced dataset on intrusion detection. I have (attack class) 3668045 samples and (benign class) 477 samples. I made a 70:30 Train test split. My problem is to predict whether the given ...
Zal's user avatar
  • 7
1 vote
0 answers
55 views

Training with extremely imbalanced Dataset

I have a object detection problem which has extremely imbalanced dataset. Lets say there is only one class to detect, say apple or not apple. This detection network will be used in a real case ...
Uce's user avatar
  • 11
1 vote
0 answers
48 views

Data Imbalance in Contextual Bandit with Thompson Sampling

I'm working with the Online Logistic Regression Algorithm (Algorithm 3) of Chapelle and Li in their paper, "An Empirical Evaluation of Thompson Sampling" (https://papers.nips.cc/paper/2011/...
MABQ's user avatar
  • 11
2 votes
1 answer
166 views

What are the possible ways to handle imbalance in multi-class image datasets?

Image imbalance is one of the major factor in the performance of DL model. Some of the methods that I found to tackle this are oversampling, under-sampling, SMOTE. Over-sampling has cons as it makes ...
Dilip C M Dept of MCA's user avatar
0 votes
3 answers
252 views

How to deal with an unbalanced dataset?

I'm constructing a feed forward neural network that predicts whether a patient will get a stroke or not. However, my dataset is very unbalanced. Out of 5111 rows, 250 contain patients that have had a ...
JanHudec's user avatar
0 votes
2 answers
49 views

How to deal with datasets which are not balanced?

I have a dataset that I want to use for training. The output of the model is a binary value (0,1) The dataset is not balanced, it has only 200 entries for output 1 and 4000 entries for output 0. When ...
mans's user avatar
  • 109
0 votes
0 answers
27 views

What's the best way to train data with unbalanced targets?

Suppose I have data I want to use for supervised learning, but there is a pretty bad target/class/labels imbalance. Should I: Limit the size of the training set to make sure there is a flat target/...
sangstar's user avatar
  • 131
0 votes
1 answer
209 views

How to arrange test dataset distribution for an imbalanced classification problem?

I have a dataset that contains 560 datapoints, and I would like to do binary classification on it. 400 datapoints belong to class 1, and 160 points belong to class 2. In the case of an imbalanced ...
Clara's user avatar
  • 11
2 votes
1 answer
261 views

How do you handle unbalanced image datasets?

I have an image data set on which I am training a CNN. The data set is slightly unbalanced. So, my solution up till now was to delete some images of the majority class. But I now realize that there ...
sahal mulki's user avatar
0 votes
1 answer
214 views

How to handle an unbalanced dataset when training object detection algorithms?

I am training an object detection model, and I have some very highly unbalanced data annotations. I have almost 11,000 images, all with dimensions of 1024 $\times$ 1024. Within those images I have the ...
ihb's user avatar
  • 129
0 votes
1 answer
166 views

How do I select the class weights for the loss function in the case of more than 2 classes?

I have a machine learning task where I would like to weight losses based on the frequency of the categorical values appearing in the data. The binary solution can be seen below, but I'd like to know ...
Jan-Samuel Wagner's user avatar
0 votes
0 answers
14 views

Evaluating a convolutional neural network on an imbalanced (academic) dataset

I have trained a posture analysis network to classify in a video of humans recorded in public places if there is a) shake-hand between two humans, b) Standing close together that their hands touch ...
kapooraae489's user avatar
2 votes
1 answer
75 views

How to handle class imbalance when the actual data are that way

My supervised learning training data are obtained from actual data; and in real cases, there's one class that happens less often than other classes, just around 5% of all cases. To be precise, the ...
Dan D.'s user avatar
  • 1,283
1 vote
1 answer
451 views

Handling imbalanced data with multiple targets

I have the model which has 3 outputs (it is a regression task, I have the angle of the steering wheel, brake and acceleration). I can divide my values to some smaller bins and in this way I can change ...
user avatar
1 vote
1 answer
201 views

Multi class text classification when having only one sample for classes

I have a dataset of texts, each text was identified with an ID number. I would like to do a prediction by finding the best match ID number for upcoming new texts. To use multi text classification, I ...
Fara's user avatar
  • 11
0 votes
1 answer
290 views

What does "adding class weights for an imbalanced dataset" mean in the case of multi-label classification?

Suppose I have the following toy data set: Each instance has multiple labels at a time. You can see I have 2 instances for Label2. However, only one instance for the other labels. It means that we ...
Nafees Ahmed's user avatar
3 votes
1 answer
84 views

How do I select the (number of) negative cases, if I'm given a set of positive cases?

We were given a list of labeled data (around 100) of known positive cases, i.e. people that have a certain disease, i.e. all these people are labeled with the same class (disease). We also have a much ...
Otto's user avatar
  • 33
2 votes
1 answer
550 views

How robust are deep networks to class imbalance?

Before deep learning, I worked with machine learning problems where the data had a large class imbalance (30:1 or worse ratios). At that time, all the classifiers struggled, even after under-sampling ...
user3667125's user avatar
  • 1,560
2 votes
2 answers
455 views

How to perform binary classification when one class is more predominant than the other?

Assuming we have big $m \times n$ input dataset, with $m \times 1$ output vector. It's a classification problem with only two possible values: either $1$ or $0$. Now, the problem is that almost all ...
Marvin's user avatar
  • 21
1 vote
1 answer
231 views

Is it possible to combine k-fold cross-validation and oversampling for a multi-class text classification task with imbalanced data?

I am dealing with an intent classification task on an Italian customer service data set. I've more or less 1.5k sentences and 29 classes (imbalanced). According to the literature, a good choice is to ...
Alfonso's user avatar
  • 65
2 votes
1 answer
248 views

How can I use Generative Adversarial Networks to solve the imbalanced class problem?

Problem setting We have to do a binary classification of data given a training dataset $D$, where most items belong to class $A$ and some items belong to class $B$, so the classes are heavily ...
frederik's user avatar
7 votes
2 answers
4k views

Does data skew matter in classification problem?

I'm working on an image classification problem using a neural network. In the training data set, 90% of the samples fall into 10% of all categories, while 10% of the sample fall into the other 90% ...
James Gan's user avatar
  • 173