1
$\begingroup$

I am using Keras to build a CNN model to classify spectograms and using the following layers:

Conv2D 32 filters, size (5,5), relu 
MaxPooling2D (2,2)  

Conv2D 32 filters, size (5,5), relu
MaxPooling2D (2,2)  

Conv2D 64 filters, size (5,5), relu
MaxPooling2D (2,2)      
                                                                  
Flatten         
Dense 64 , relu                                                        
Dense 32 , relu                                                     
Dense 5 (softmax)

With this model I was able to comfortably achieve more than 97% training accuracy and around 84% validation accuracy. Unfortunately, I have very few data samples (slightly more than 1200), which is my main suspicion on why the model is overfitting.

As I am not able to get more data samples, I opted to use SpecAugmentation. I used to get all my classes (5) to 1000 samples each, meaning I generated around 3800 samples.

After I fit the model to this augmented data I got more than 97% validation data, which I found very strange. I am thinking that I am introducing bias because I am performing the data augmentation before splitting into training and validation sets, which may lead to having the same sample in both sets (obviously they are not 100% equal but one is derived from the other).

Should I only be performing data augmentation after splitting? If I use data augmentation only on the training set will I not be overfitting to the training set as well, as it will see very similar samples several times?

What other techniques would you recommend for data augmentation besides SpecAugmentation?

Thanks

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

You are right you should apply the augmentation after splitting. The goal of the validation data is to assess the results on data that has not been seen during training. When doing so you way have different versions of the same sample in the train and test splits.

The goal of SpecAugment is to make the model learn from different part of the audio. Time and frequency masking hide some information so the model will try to build its decision based on the other parts of the spectrogram.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thank you for your validation. Any idea on how I could prevent the overfitting of the CNN with the small dataset I am using? I am still exploring BatchNormalization or even changing the batch size, for example, but I am not sure if this will fix the overfitting problem. Any suggestions? $\endgroup$
    – Tomás Nunes
    Jul 15, 2022 at 12:23
  • $\begingroup$ SpecAugment and batch normalization should help. Maybe a smaller batch size and lower learning rate can help, what optimizer algorithm are you using? I recently tried flipping the audios along the time axis (you can consider it as playing it in reverse) and it helped me a bit. It is also possible to randomly crop a part of the audios, although it never helped me. $\endgroup$
    – theophile
    Jul 18, 2022 at 6:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .