1
$\begingroup$

I have a dataset of 3000 8x8 images, and I would like to train a GAN for an image generation purpose.

I am planning to start with a simple GAN model and see if it overfits. Before training, I try to do a comparison of the discriminator model prediction using real image input against the whole GAN model prediction using random seed input. My thought process is that since this model is not trained yet, the output for real images and fake images by the discriminator should not be predictable.

However, the discriminator model prediction using real image input always returns a value very close to 1.0, and the whole GAN model prediction using random seed input always returns a value near 0.5 with a small deviation. I suspect that during training, the model would simply pull the 0.5 value near 0.0 and would never actually learn from the dataset.

I try to increase the training parameters and different initializers, but the output is still the same.

By ruling out the possibility a bad dataset, what could be the reason for this situation?

This is some sneak peek of the generator and discriminator model building: https://pastebin.com/ehMDP7k6

$\endgroup$
4
  • $\begingroup$ what do you mean by "a comparison of the discriminator model prediction ... against the whole GAN model prediction"? do you mean the output of your generator for fake and real images? $\endgroup$ – Aray Karjauv Nov 16 '20 at 19:29
  • $\begingroup$ In other words, I feed real images array into the discriminator model, and it outputs values close to 1.0. Next, I feed fake images (the one that is generated by the generator) into the discriminator model, and it outputs values close to 0.5. These happens too often, and it is very predictable. $\endgroup$ – dee cue Nov 17 '20 at 11:18
  • $\begingroup$ Did you try to generate fake images? Since the main goal of GANs to produce synthetic samples, the main matric will be the quality of generated images. Take a look at this answer. There is a plot of G loss and D loss of DCGAN. We can see that D loss is near 0, whereas G loss has oscillations, but G was able to produce good samples. $\endgroup$ – Aray Karjauv Nov 17 '20 at 11:29
  • $\begingroup$ Yes, and it only generates noises. Although, I do notice some preference of a certain pallete, and I think this is a good start. The D accuracy went constant at 0.5 after 20 epochs, and the G accuracy went up to 1.0, and both losses gradually went down. All metrics did oscillate in the middle of the training but in small amount of deviation. Did my model overfit at epoch 20? $\endgroup$ – dee cue Nov 17 '20 at 11:54
2
$\begingroup$

I took a look at your model. It seems you have incorrect architecture. The Conv2D layers in your D should have following params: (n_filters, kernel=3, padding='same'), where n_filters is the number of filters and it usually should be doubled as per DCGAN architecture. You can also use strides but since your images are small it won't make any sense.

The D network also should not include UpSampling2D layer. This layer can be used in G network instead of Conv2DTranspose.

Your images should be normalized to the range of [-1, 1] and the activation function of G should be tanh.

You also included m.add(keras.layers.Reshape((8, 8, 3))) to your G network which is the final size of your data. Since you have the final size in the first layers you don't have to include upsampling Conv2DTranspose layers.

Your final model should look like

def make_discriminator_model():
    model = tf.keras.Sequential()
    model.add(layers.Conv2D(8, kernel=3, strides=1, padding='same',
                                     input_shape=[8, 8, 1]))
    model.add(layers.LeakyReLU())
    model.add(layers.Dropout(0.3))

    model.add(layers.Conv2D(16, kernel=3, strides=1, padding='same'))
    model.add(layers.LeakyReLU())
    model.add(layers.Dropout(0.3))

    model.add(layers.Flatten())
    model.add(layers.Dense(1))

    return model

def make_generator_model():
    model = tf.keras.Sequential()
    model.add(layers.Dense(8*8*16, use_bias=False, input_shape=(100,)))
    model.add(layers.BatchNormalization())
    model.add(layers.LeakyReLU())

    model.add(layers.Reshape((8, 8, 16)))

    model.add(layers.Conv2D(8, kernel=3, strides=1, padding='same'))
    model.add(layers.BatchNormalization())
    model.add(layers.LeakyReLU())

    model.add(layers.Conv2D(1, kernel=3, strides=1, padding='same', activation='tanh'))

    return model

Note that I did not test the model, so you should adapt it to the shape of your data.

I encourage you to follow this tutorial.

Update regarding overfitting

GANs do not have overfitting problem in the classical sense. Instead, vanilla GAN have other problems like vanishing gradient. It happens when the D becomes overconfident regarding fake samples. In that case, D stops providing useful information to the G, since its error becomes 0. To find out if training fell into this problem, you should plot D and G loss. It will look as follows:

DCGAN loss

Another problem is the mode collapse. In that case, the G tricks the D by producing only one type of samples which looks realistic but G won't represent the real data distribution. Therefore, the generated samples will be homogeneous.

For more information, see Improved Techniques for Training GANs and Wasserstein GAN.

$\endgroup$
2
  • $\begingroup$ Thank you for the guidance. I certainly built my models with baseless knowledge, and most importantly, I forgot to normalise my image array before feeding into the discriminator, which would explain a lot at my initial problem. I accepted this answer for raising the normalisation problem, and upvoted for the extra guidance. $\endgroup$ – dee cue Nov 17 '20 at 14:38
  • $\begingroup$ I forgot to mention the overfitting issue. I updated the answer accordingly $\endgroup$ – Aray Karjauv Nov 17 '20 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.