I'm currently trying to train a GAN to recreate similar images from a dataset. The dataset is using the Eiffel Tower Pictures from Googles Quick Draw dataset. The images aren't very large (only 12x12 pixels) and are all black and white.
The performance increases at an expected rate initially however after a certain point the quality begins to go down despite the cost from both generator and discriminator networks seeming to stay at a steady value as expected.
I've tried changing the learning rate and other hyper parameters however they all end up with the same result of the network eventually getting worse.
My learning rate is currently 0.01 which I'm aware is quite high compared to what other people are using but anything lower takes too long to train and the results don't seem any better even if I do give it long enough.
Any pointers on what could be causing this or the specific name of this problem if it's a common issue with GANs would be appreciated.