I am training a semi-supervised GAN network using data from multiple subjects. I separated the labeled and unlabeled data based on my subjects, so there is no leakage, while having much more unlabeled data than labeled data. After few epochs training accuracy hits 100% which normally indicates overfitting, however the performance on the validation and test sets keeps increasing for 200-300 epochs. Is this considered overfitting and is there an explanation for this behavior?

  • $\begingroup$ There is a lot of variables here which can help answer this. The main one: How are you measuring accuracy? For example, if you measure a correct prediction as above 0.5 when positive and below 0.5 when negative, the accuracy could be 100% well before overfitting, as the network still has a lot of room to approach 1 and 0. $\endgroup$
    – Recessive
    Oct 14 at 3:30
  • $\begingroup$ I am in a multiclass setting with k=12 classes and I use argmax on the output logits, and compare them to real labels using sklearn's accuracy function. I am also computing the loss, for example in one epoch i had 0.4 for validation and 0.01 for training, and later training loss drops to 1e-5 while validation loss oscillates around 1, with no noticable impact on the accuracy score. $\endgroup$
    – Hazar
    Oct 14 at 11:50
  • $\begingroup$ How does it do on the test set? This does sound like it's overfitting based on the loss. I would triple check you are definitely running the same version of the network that's producing this loss when computing accuracy before continuing, because it does sound quite strange that loss would increase by over 100% with accuracy also increasing at the same time $\endgroup$
    – Recessive
    Oct 15 at 1:05
  • $\begingroup$ Same on the test set, performances kept increasing even after signs of overfitting on the training set. For example in a recent model, I obtained 100% acc, 0.0016 loss for training, around 93% acc and 0.4 loss for both validation and test sets that came from different subjects. $\endgroup$
    – Hazar
    Oct 16 at 2:06
  • $\begingroup$ Can you maybe post a graph of the loss and accuracy for training and validation? That could help see what's going on here more exactly. $\endgroup$
    – Recessive
    Oct 16 at 11:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.