# What to do with a GAN that trained well but got worse over time?

I am training a WGAN-GP network based on the following paper, though I am using a different dataset. Now, for the first ~ 60-70 epochs, my network trained really well, which I could see in the loss going down, but I also made sure to regularly check the quality of the images.

Unfortunately, what I am seeing now (for the last $$20$$ epochs) is that the generator is getting worse and worse, the images don't look that good anymore. I save checkpoints every epochs, so in principle, I could stop training and get myself a state of the network from where it was still performing quite okay.

However, my question would be: How can I improve the training of the GAN? Would you decrease the learning rate?

I use a batch size of 124 and a learning rate of 1e-3. Maybe I could/should continue training (with a checkpoint that was still quite okay) with a learning rate of 5e-4?

Any other hints would be appreciated!

• Hi Mafu, thanks! (i) You're right, the LR of $1e-3$ seems to high.. Maybe I will restart training from scratch and see what it does. (ii) Interesting, I didn't know one could also use Dropout with GANs.. (iii) My critic (as for WGANs, we don't strictly have a discriminator) updates itself $5$-times before a generator update. (iv) I have enough training samles (more than $200$k images). (v) My network architectures are heavy, I won't deny that.. I oriented myself at this: github.com/aladdinpersson/Machine-Learning-Collection/blob/… – Anonymous5638 Mar 16 at 8:17