Questions tagged [imbalanced-datasets]
For questions that involve imbalanced (or unbalanced) datasets.
28
questions
1
vote
1
answer
47
views
In multi-class classification, how accurate can the model be if there's class imbalance?
My dataset has essentially multi-classification problem, where I have the treatment failure (0), cure (1) and relapses (3) of patients that are associated with a series of covariates (~100 different ...
- 13
0
votes
0
answers
23
views
Is it necessary that the number of samples of one class be balanced with other classes in a classification problem?
Consider a classification problem using machine learning techniques (e.g. malware detection). In such a problem, is it necessary that the number of samples from each class (in the mentioned example, ...
- 11
0
votes
1
answer
60
views
Why would balancing be so helpful when the imbalance is minimal?
I have a binary classification problem with a modest-to-none class imbalance (33% positive class-66% negative class). When I don't impose class balance, my XGBoost model produces no positive class ...
0
votes
0
answers
19
views
Why my deep learning model (FCNN/ 1DCNN) fails to learn when training on medical dataset?
I am working on a project to predict the severity of the disease, Hemophilia using a deep learning model(FCNN or 1DCNN). I am working based on the information provided in this article: https://www....
- 1
0
votes
1
answer
60
views
How to evaluate binary classifier on imbalanced dataset?
I have trained a Decision Tree model on an imbalanced dataset. I got the following results for the test set from the sklearn and imblearn classification reports (attached below). Moreover, the other ...
- 7
1
vote
1
answer
67
views
Fine tuning a Deep Learning model post training
I have trained a CNN in a binary classification problem, however the original problem has 6 different classes, of which, I am only interested in classifying one, so if it is that certain class or not....
- 121
0
votes
1
answer
152
views
How to interpret binary classification metrics on an imbalanced data set?
I have an imbalanced dataset on intrusion detection. I have (attack class) 3668045 samples and (benign class) 477 samples. I made a 70:30 Train test split. My problem is to predict whether the given ...
- 7
1
vote
0
answers
73
views
Training with extremely imbalanced Dataset
I have a object detection problem which has extremely imbalanced dataset. Lets say there is only one class to detect, say apple or not apple. This detection network will be used in a real case ...
- 11
1
vote
0
answers
58
views
Data Imbalance in Contextual Bandit with Thompson Sampling
I'm working with the Online Logistic Regression Algorithm (Algorithm 3) of Chapelle and Li in their paper, "An Empirical Evaluation of Thompson Sampling" (https://papers.nips.cc/paper/2011/...
- 11
2
votes
1
answer
173
views
What are the possible ways to handle imbalance in multi-class image datasets?
Image imbalance is one of the major factor in the performance of DL model. Some of the methods that I found to tackle this are oversampling, under-sampling, SMOTE. Over-sampling has cons as it makes ...
0
votes
3
answers
338
views
How to deal with an unbalanced dataset?
I'm constructing a feed forward neural network that predicts whether a patient will get a stroke or not. However, my dataset is very unbalanced. Out of 5111 rows, 250 contain patients that have had a ...
- 9
0
votes
2
answers
59
views
How to deal with datasets which are not balanced?
I have a dataset that I want to use for training.
The output of the model is a binary value (0,1)
The dataset is not balanced, it has only 200 entries for output 1 and 4000 entries for output 0.
When ...
- 109
0
votes
0
answers
28
views
What's the best way to train data with unbalanced targets?
Suppose I have data I want to use for supervised learning, but there is a pretty bad target/class/labels imbalance. Should I:
Limit the size of the training set to make sure there is a flat target/...
- 131
0
votes
1
answer
242
views
How to arrange test dataset distribution for an imbalanced classification problem?
I have a dataset that contains 560 datapoints, and I would like to do binary classification on it. 400 datapoints belong to class 1, and 160 points belong to class 2. In the case of an imbalanced ...
- 11
2
votes
1
answer
308
views
How do you handle unbalanced image datasets?
I have an image data set on which I am training a CNN. The data set is slightly unbalanced. So, my solution up till now was to delete some images of the majority class.
But I now realize that there ...
- 45
0
votes
1
answer
276
views
How to handle an unbalanced dataset when training object detection algorithms?
I am training an object detection model, and I have some very highly unbalanced data annotations. I have almost 11,000 images, all with dimensions of 1024 $\times$ 1024.
Within those images I have the ...
- 129
0
votes
1
answer
173
views
How do I select the class weights for the loss function in the case of more than 2 classes?
I have a machine learning task where I would like to weight losses based on the frequency of the categorical values appearing in the data. The binary solution can be seen below, but I'd like to know ...
0
votes
0
answers
14
views
Evaluating a convolutional neural network on an imbalanced (academic) dataset
I have trained a posture analysis network to classify in a video of humans recorded in public places if there is a) shake-hand between two humans, b) Standing close together that their hands touch ...
2
votes
1
answer
79
views
How to handle class imbalance when the actual data are that way
My supervised learning training data are obtained from actual data; and in real cases, there's one class that happens less often than other classes, just around 5% of all cases.
To be precise, the ...
- 1,293
1
vote
1
answer
557
views
Handling imbalanced data with multiple targets
I have the model which has 3 outputs (it is a regression task, I have the angle of the steering wheel, brake and acceleration). I can divide my values to some smaller bins and in this way I can change ...
user40943
1
vote
1
answer
206
views
Multi class text classification when having only one sample for classes
I have a dataset of texts, each text was identified with an ID number. I would like to do a prediction by finding the best match ID number for upcoming new texts. To use multi text classification, I ...
- 11
0
votes
1
answer
320
views
What does "adding class weights for an imbalanced dataset" mean in the case of multi-label classification?
Suppose I have the following toy data set:
Each instance has multiple labels at a time.
You can see I have 2 instances for Label2. However, only one instance for the other labels. It means that we ...
- 41
3
votes
1
answer
85
views
How do I select the (number of) negative cases, if I'm given a set of positive cases?
We were given a list of labeled data (around 100) of known positive cases, i.e. people that have a certain disease, i.e. all these people are labeled with the same class (disease). We also have a much ...
- 33
2
votes
1
answer
673
views
How robust are deep networks to class imbalance?
Before deep learning, I worked with machine learning problems where the data had a large class imbalance (30:1 or worse ratios). At that time, all the classifiers struggled, even after under-sampling ...
- 1,660
2
votes
2
answers
504
views
How to perform binary classification when one class is more predominant than the other?
Assuming we have big $m \times n$ input dataset, with $m \times 1$ output vector. It's a classification problem with only two possible values: either $1$ or $0$.
Now, the problem is that almost all ...
- 21
1
vote
1
answer
239
views
Is it possible to combine k-fold cross-validation and oversampling for a multi-class text classification task with imbalanced data?
I am dealing with an intent classification task on an Italian customer service data set.
I've more or less 1.5k sentences and 29 classes (imbalanced).
According to the literature, a good choice is to ...
- 65
2
votes
1
answer
271
views
How can I use Generative Adversarial Networks to solve the imbalanced class problem?
Problem setting
We have to do a binary classification of data given a training dataset $D$, where most items belong to class $A$ and some items belong to class $B$, so the classes are heavily ...
- 23
7
votes
2
answers
4k
views
Does data skew matter in classification problem?
I'm working on an image classification problem using a neural network. In the training data set, 90% of the samples fall into 10% of all categories, while 10% of the sample fall into the other 90% ...
- 173