0
$\begingroup$

I am having this issue where the generated images get better after a while, but just keep getting worse and worse. I monitored the gradient coming into the generator for the picture of "3":enter image description here

This was at the start of training. However, as training progressed, the generator learnt from this gradient and was about to make better images when the gradient just started getting worse: enter image description here

The generated image just started getting worse and worse : enter image description here

Now its just 4 lines on the screen that don't change at all. I'm starting to guess that the critic learnt the filters for the generated image instead of the real image and is unable to instruct the generator how to generate proper images because it stopped learning from the real images after a while. I may be misunderstanding something.

These are my models :

def make_generator_model():
    model = tf.keras.Sequential()
    model.add(layers.Dense(7*7*256, use_bias=False, input_shape=(100,)))
    model.add(layers.LeakyReLU())
    model.add(layers.Reshape((7, 7, 256)))
    assert model.output_shape == (None, 7, 7, 256)
    model.add(layers.Conv2DTranspose(128, (5, 5), strides=(1, 1), padding='same', use_bias=False))
    assert model.output_shape == (None, 7, 7, 128)
    model.add(layers.LeakyReLU())
    model.add(layers.Conv2DTranspose(64, (5, 5), strides=(2, 2), padding='same', use_bias=False))
    assert model.output_shape == (None, 14, 14, 64)
    model.add(layers.LeakyReLU())
    model.add(layers.Conv2DTranspose(1, (5, 5), strides=(2, 2), padding='same', use_bias=False, activation='tanh'))
    assert model.output_shape == (None, 28, 28, 1)
    return model

def make_discriminator_model():
    model = tf.keras.Sequential()
    model.add(layers.Conv2D(64, (5, 5), strides=(2, 2), padding='same', input_shape=[28, 28, 1]))
    model.add(layers.LeakyReLU())
    model.add(layers.Conv2D(128, (5, 5), strides=(2, 2), padding='same'))
    model.add(layers.LeakyReLU())
    model.add(layers.Flatten())
    model.add(layers.Dense(1))
    return model

I am just using the standard loss functions for my gan(WGAN with spectral norm). Why do you think this happens? Am I misunderstanding something?

Edit : This is one of the images from the dataset for the picture of 3 : enter image description here

Also, I am using a learning rate of 0.0005 for both the generator and the critic, and am using a self-made ML-framework.

While training the critic, i am updating the the critic based on a batch of both fake and real images in a single batch. Maybe since the gradients are getting averaged before updating the critic, it is learning the wrong filters?

$\endgroup$
4
  • $\begingroup$ Can you give more details on your training and data? You showed an image, but i dont know what it should look like, so i cannot judge whether it is failling or not. $\endgroup$ Commented May 28 at 7:41
  • $\begingroup$ The images that i am training on are 3's from the mnsit dataset. The generator just gets stuck on 4 horizontal lines everytime when training on 3's(everytime i train it). When training on 9's, it gets stuck on 3 vertical lines(everytime). Why this happens, i do not know. I edited the answer to show one of the target images that i want the generator to be able to learn $\endgroup$ Commented May 28 at 7:48
  • $\begingroup$ At the start, the discriminator's gradients fit the general shape of a 3, and almost become the exact shape of a 3. But, the generator learns from this and gets a little bit better, and then the gradients coming in from the critic just become worse and just horizontal lines--> and then the generator starts producing horizontal lines. I am unable to debug this issue $\endgroup$ Commented May 28 at 7:54
  • $\begingroup$ Please use edit to reword the start, since this looks like your "answer" to another question copied over to a question. It's OK to add a link to the related question, but otherwise your question should be stand-alone. I think in this case you could achieve that by quickly editing the first part so it doesn't seem to be following on from anything else $\endgroup$ Commented May 28 at 8:12

1 Answer 1

-1
$\begingroup$

It seems like spectral norm is applied to your WGAN's critic along with Gradient penalty. Nobody online(afaik) has been able to get a vanilla WGAN with Spectral normalization working. You need gradient penalty or something like that in addition to spectral norm, which is kinda confusing as many people try to sell it as a replacement to other methods

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .