I have a dataset of 3000 8x8 images, and I would like to train a GAN for an image generation purpose.
I am planning to start with a simple GAN model and see if it overfits. Before training, I try to do a comparison of the discriminator model prediction using real image input against the whole GAN model prediction using random seed input. My thought process is that since this model is not trained yet, the output for real images and fake images by the discriminator should not be predictable.
However, the discriminator model prediction using real image input always returns a value very close to 1.0, and the whole GAN model prediction using random seed input always returns a value near 0.5 with a small deviation. I suspect that during training, the model would simply pull the 0.5 value near 0.0 and would never actually learn from the dataset.
I try to increase the training parameters and different initializers, but the output is still the same.
By ruling out the possibility a bad dataset, what could be the reason for this situation?
This is some sneak peek of the generator and discriminator model building: https://pastebin.com/ehMDP7k6