Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [overfitting]

For questions related to the concept of overfitting in machine learning, which can be loosely defined as the gap between the performance on the training set and the performance on the test set.

Filter by
Sorted by
Tagged with
0 votes
0 answers
29 views

Attention module (CBAM) in CNN tend to saturate values to 1

In the context of image classification, I am using a feature extractor based on a resnet-like architecture (ResNet12): four residual blocks, each of which is made of two consecutive conv3x3, batch ...
Lorenzo's user avatar
0 votes
2 answers
110 views

Why does model overfitting lead to poor generalization?

If a model overfit to the training data, why does it generalize poorly? Consider the basic problem of a noisy 2d dataset where I am fitting polynomials. A good model would be a parabola and a line ...
JobHunter69's user avatar
0 votes
0 answers
29 views

High Fluctuations in Validation Curve

Below I attach an image of accuracy curves. I got a lot of suggestions regarding some improvement in below curves. Following are my experiments in order to make the curve stable--> I used lr = 4....
Sarvagya Porwal's user avatar
0 votes
0 answers
7 views

How much is the acceptable percentage for Random Forest in Landslides prediction?

RF had been developed to overcome overfitting in decision trees but in some cases RF still experiences overfitting in landslide prediction, which varies from 2% to 12%. How much overfitting is ...
DOROTHY's user avatar
1 vote
0 answers
17 views

Failing to train and avoid overfit on noisy training data

I have added some Gaussian noise to CIFAR10 training and test set. I am using VGG16 and ResNet34 as the model to be used for training. Under normal training conditions, where the standard CIFAR10 is ...
StudentV's user avatar
0 votes
1 answer
120 views

MobileNet validation loss not decreasing over time

I am trying to train a MobileNetV2 on a custom dataset, to image Classification task. Cardinality is 864 images, split in 70%/20%/10%, balanced between the 3 different classes. Weights are pre-loaded ...
elbarto's user avatar
1 vote
1 answer
143 views

How does deep learning overcome overfitting?

From Berkeley CS182, SP22: https://cs182sp22.github.io/assets/lecture_slides/2022.01.26-ml-review-pt2.pdf. Can someone help me interpret this diagram? I understand the graph on the left, but I don't ...
9j09jf02jsd's user avatar
0 votes
1 answer
58 views

What are possible reasons for the validation loss increasing with more data?

I trained a neural network on an NLP problem and compared the loss and BLEU score on the validation data with the same training parameters in two scenarios: a) when I trained on 25% of the data, b) ...
postnubilaphoebus's user avatar
1 vote
0 answers
217 views

What is wrong with my PyTorch model training on CIFAR10?

I am training a ResNet model on CIFAR10 dataset. For the training subset, I selected a random 1% of the train data from the default train/test split. For the test subset I used the whole default test ...
Liisjak's user avatar
  • 11
0 votes
1 answer
98 views

Does this look like overfitting?

I'm using a Decision Tree that gave me great test metrics. Then I checked the learning curve, but it seems a little strange to me regarding the training score. Do you think there is a problem with ...
giovasbr's user avatar
0 votes
2 answers
736 views

What should I do if my validation score is good, but my test score is bad?

I've trained my artificial neural network, and, as per standard practice, I've picked out the one neural network throughout training that did the best on my validation dataset. That is, the neural ...
Pro Q's user avatar
  • 103
0 votes
2 answers
290 views

How do you interpret this train vs test accuracy scores? is the model under or over fitting?

What does this difference in train and test accuracy mean?
ProgrammingBot's user avatar
0 votes
0 answers
20 views

How to speed up the learning process

I have built a network that performans pretty well on my data. The issue I have is that for a larger number of epochs at the start of the training process the val/train acc/loss are stagnating (for ...
Skobo Do's user avatar
1 vote
1 answer
118 views

fondamental question about regularization techniques to solve overfitting problem in neural networks

I have a text classification neural network based on BERT that overfits. The accuracy on the training dataset is 95%, whereas it is 68% on the validation dataset. Using some classical regularization ...
tammuz's user avatar
  • 113
0 votes
1 answer
73 views

How do I know if my Random Forest Regressor Model is overfitted?

Im creating a Random Forest Regressor Model with a small dataset (30 data points). I tried with other models but RF was the best one, however, after applying GridSearchCv I got that the training set ...
Gaby's user avatar
  • 3
7 votes
2 answers
799 views

Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?

I plan to use my predictions as ground truth to continue training my model. These predictions are of course reviewed during this process. Is there an argument against that (reinforcement of slight ...
thzu's user avatar
  • 73
1 vote
0 answers
72 views

Why does the SVM perform poorly on test data that has a different class distribution than the training data?

Do you know why the SVM performs poorly on test data that has a different class distribution than the training data? The training data has around 15 classes, and the additional testing data has around ...
Allie's user avatar
  • 11
4 votes
2 answers
316 views

How does Regularization Reduce Overfitting?

As I understand, this is the general summary of the Regularization-Overfitting Problem: The classical "Bias-Variance Tradeoff" suggests that complicated models (i.e. models with more ...
stats_noob's user avatar
3 votes
1 answer
279 views

Can neural networks learn noise?

I'm interested in the following graphs. A neural network was trained to recognise digits from the MNIST dataset and then the labels were randomly shuffled and the following behaviour was observed. ...
Featherball's user avatar
0 votes
2 answers
164 views

Validation set performance increasing even after seemingly overfit on training set

I am training a semi-supervised GAN network using data from multiple subjects. I separated the labeled and unlabeled data based on my subjects, so there is no leakage, while having much more unlabeled ...
Hazar's user avatar
  • 31
2 votes
0 answers
21 views

2-stage model overfitting

I'm trying to build an entity matching model. There are 2 kinds of features - binary (0/1) and text features. Initially I made a deep learning model that uses character level embeddings of some of the ...
user9343456's user avatar
0 votes
0 answers
105 views

What can cause massive instability in validation loss?

I'm working with very weird data that is apparently very hard to fit. And I've noticed a very strange phenomenon where it can go from roughly 0.0176 validation MSE to 1534863.6250 validation MSE in ...
profPlum's user avatar
  • 424
2 votes
1 answer
643 views

Does a bigger neural network learn "worse" representations than a small neural network when the amount of data isn't enough?

Assume we have a neural network and we want to train it on a classification problem. The hidden layers of the neural network are kind of feature representations of the input data. If the neural ...
realmarv's user avatar
0 votes
0 answers
566 views

Why is BatchNormalization causing severe overfitting to my data?

So I've been making a mini version of VGGNet, trying to tweak the hyperparameters to match the CIFAR-100 dataset. It was running slow at first but I was able to get decent accuracy after 60 epochs or ...
Christopher Centrella's user avatar
1 vote
1 answer
1k views

What does it mean by overfitting the test set?

Consider the following statement from p14 of Naive Bayes and Sentiment Classification While the use of a devset avoids overfitting the test set, having a fixed training set, devset, and test set ...
hanugm's user avatar
  • 3,880
2 votes
1 answer
2k views

Should I continue training if the neural network attains 100% training accuracy?

I have a neural network where there are two hidden layers. Each hidden layer has 128 neurons. The input layer has 20 inputs, and the output layer has 3 outputs. I have 1 million records of data. 80% ...
user366312's user avatar
1 vote
1 answer
53 views

What are the 'noisy factors' leading to overfitting?

Consider the following excerpt from section 5.5 Regularization (p. 13) of this chapter Logistic Regression. There is a problem with learning weights that make the model perfectly match the training ...
hanugm's user avatar
  • 3,880
0 votes
1 answer
451 views

Is it possible to overfit a model on infinite amounts of data?

This is a theoretical question. Is it possible to overfit a model on infinite amounts of data? Let me clarify there are no duplicates. Say, we have a generator function that produces data, with the ...
ToAskOrNotToAsk's user avatar
3 votes
2 answers
146 views

When exactly am I overfitting -- contradicting metrics

I am training an object detection machine learning pipeline. Among the many metrics provided out of the box by tensorflow object detection API, I look at total_loss and DetectionBoxes_Precision/mAP@....
user1091141's user avatar
0 votes
0 answers
104 views

Identifying if a model is over or under-fitting via graphs

I am working on a Neural Network and have plotted the performance of my model. However the plots seem not to fit the "trends" (which help you identify the issue with your model) presented in ...
jr123456jr987654321's user avatar
0 votes
2 answers
960 views

Do larger numbers of hidden layers have a bigger effect on a classification model's accuracy?

I trained different classification models using Keras with different numbers of hidden layers and the same number of neurons in each layer. What I found was the accuracy of the models decreased as the ...
Shonix3373's user avatar
1 vote
0 answers
51 views

How does the loss landscape look like or change when a model is overfitting?

My understanding is that when a model starts overfitting, it no longer learns useful features and starts remembering the training data set. Given enough epochs and sufficient parameters, a model can ...
user289602's user avatar
3 votes
1 answer
133 views

Does adding a model complexity penalty to the loss function allow you to skip cross-validation?

It's my understanding that selecting for small models, i.e. having a multi-objective function where you're optimizing for both model accuracy and simplicity, automatically takes care of the danger of ...
Redrock's user avatar
  • 33
1 vote
1 answer
586 views

Can residual connections be beneficial when we have a small training dataset?

I have a classification problem, for which an inadequate amount of training data is available. Also, there is no known practical data augmentation approach for this problem (as no unlabelled data is ...
Reza_va's user avatar
  • 89
0 votes
1 answer
234 views

Is it possible that the model is overfitting when the training and validation accuracy increase?

I am aware of similar questions that have been asked, and I have gone through many. I want to bring my case to SE to understand better what my results are. I am working with a large dataset (around ...
nachofest's user avatar
0 votes
1 answer
92 views

What are possible ways to combat overfitting or improve the test accuracy in my case?

I have asked a question here, and one of the comments suggested that this is a case of severe overfitting. I made a neural network, which uses residual boosting (which is done via a KNN), and I am ...
jr123456jr987654321's user avatar
0 votes
2 answers
94 views

Could I just choose the other (non-predicted) class when the accuracy is low?

I have a binary classification problem. My neural network is getting between 10% and 45% accuracy on the validation set and 80% on the training set. Now, if I have a 10% accuracy and I just take the ...
jr123456jr987654321's user avatar
3 votes
1 answer
382 views

When using PCA for dimensionality reduction of the feature vectors to speed up learning, how do I know that I'm not letting the model overfit?

I'm following Andrew Ng's course for Machine Learning and I just don't quite understand the following. Using PCA to speed up learning Using PCA to reduce the number of features, thus lowering the ...
AfiJaabb's user avatar
  • 131
1 vote
0 answers
30 views

Dealing with bias in multi-channel auto encoders

The problem I have a multi-channel 1D signal I want to auto-encode. I am unable to resonstruct the input when the number of channels increases. Code I am using a convolutional encoder, and a ...
Gulzar's user avatar
  • 759
1 vote
0 answers
93 views

Underfitting a single batch: Can't cause autoencoder to overfit multi-sample batches of 1d data. How to debug?

TL;DR I am unable to overfit batches with multiple samples using autoencoder. Fully connected decoder seems to handle more samples per batch than conv decoder, but then also fails when number of ...
Gulzar's user avatar
  • 759
8 votes
3 answers
10k views

How should we regularize an LSTM model?

There are five parameters from an LSTM layer for regularization if I am correct. To deal with overfitting, I would start with reducing the layers reducing the hidden units Applying dropout or ...
Leo's user avatar
  • 133
1 vote
2 answers
780 views

Why do the training and validation loss curves diverge?

I was training a CNN model on TensorFlow. After a while I came back and saw this loss curve: The green curve is training loss and the gray one is validation loss. I know that before epoch 394 the ...
Sepehr Golestanian's user avatar
1 vote
0 answers
27 views

React on train-validation curve after trening

I have a regression task that I tray to solve with AI. I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance) I understand basic ...
Marko Zadravec's user avatar
2 votes
2 answers
983 views

Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?

Background From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or ...
Kevin's user avatar
  • 133
1 vote
0 answers
123 views

How can I reconstruct sparse one-hot encodings using an RBM?

I am currently working with a categorical-binary RBM, where there are 50 categorical visible units and 25 binary hidden units. The categorical visible units are expressed in one-hot encoding format, ...
mhdadk's user avatar
  • 243
1 vote
1 answer
482 views

Why does the accuracy drop while the loss decrease, as the number of epochs increases?

I've been trying to find the optimal number of epochs that I should train my neural network (that I just implemented) for. The visualizations below show the neural network being run with a variable ...
eGood's user avatar
  • 11
1 vote
1 answer
1k views

How much overfitting is acceptable?

I have a deep learning configuration in which I obtain good results on the validation set but even better results in the training set. From my understanding this means that there is overfitting to ...
Tony Reis's user avatar
0 votes
2 answers
613 views

How to avoid over-fitting using early stopping when using R cross validation package caret

I have a data set with 36 rows and 9 columns. I am trying to make a model to predict the 9th column I have tried modeling the data using a range of models using caret to perform cross-validation and ...
user1573820's user avatar
2 votes
1 answer
91 views

Why are large models necessary when we have a limited number of training examples?

In Goodfellow et al. book Deep Learning chapter 12.1.4 they write These large models learn some function $f(x)$, but do so using many more parameters than are necessary for the task. Their size is ...
Borun Chowdhury's user avatar
1 vote
0 answers
239 views

How to overfit GANs with a single image

When designing CNN for image recogition a commonly used sainty check to see if a model is working/designed fine is to see if we are able to overfit the model with a very small subset of images. I am ...
ArunJose's user avatar
  • 111