0
$\begingroup$

After following this interesting collection of tutorials for GANs

https://www.youtube.com/playlist?list=PLhhyoLH6IjfwIp8bZnzX8QR30TRcHO8Va

I've been playing around experimenting Wasserstein GAN with gradient penalty.

Despite this, I'm still wondering about the correct interpretation for the loss function of it. The WGAN loss should be more interpretable as it outputs numbers both for generator and discriminator. So I guess, the lower these numbers, the better the performance. But, actually training such a model on MNIST dataset for few of the first batches these are the results that I get:

Epoch [0/5] Batch 0/938               Loss D: -10.0764, loss G: 0.0398
Epoch [0/5] Batch 1/938               Loss D: -10.2070, loss G: 0.1098
Epoch [0/5] Batch 2/938               Loss D: -10.3934, loss G: 0.1924
Epoch [0/5] Batch 3/938               Loss D: -10.6212, loss G: 0.3100
Epoch [0/5] Batch 4/938               Loss D: -10.8799, loss G: 0.4550
Epoch [0/5] Batch 5/938               Loss D: -11.1498, loss G: 0.6151
Epoch [0/5] Batch 6/938               Loss D: -11.3589, loss G: 0.7694
Epoch [0/5] Batch 7/938               Loss D: -11.6440, loss G: 0.9438
Epoch [0/5] Batch 8/938               Loss D: -11.9077, loss G: 1.1210
Epoch [0/5] Batch 9/938               Loss D: -12.1325, loss G: 1.2562
Epoch [0/5] Batch 10/938               Loss D: -12.3636, loss G: 1.4303
Epoch [0/5] Batch 11/938               Loss D: -12.6063, loss G: 1.5649
Epoch [0/5] Batch 12/938               Loss D: -12.8531, loss G: 1.7029
Epoch [0/5] Batch 13/938               Loss D: -13.0316, loss G: 1.8446
Epoch [0/5] Batch 14/938               Loss D: -13.2957, loss G: 1.9719
Epoch [0/5] Batch 15/938               Loss D: -13.5424, loss G: 2.1089
Epoch [0/5] Batch 16/938               Loss D: -13.7459, loss G: 2.2336
Epoch [0/5] Batch 17/938               Loss D: -13.9798, loss G: 2.3476
Epoch [0/5] Batch 18/938               Loss D: -14.1752, loss G: 2.4605
Epoch [0/5] Batch 19/938               Loss D: -14.4181, loss G: 2.5755
Epoch [0/5] Batch 20/938               Loss D: -14.6235, loss G: 2.6800
Epoch [0/5] Batch 21/938               Loss D: -14.8427, loss G: 2.7787
Epoch [0/5] Batch 22/938               Loss D: -14.9991, loss G: 2.8748
Epoch [0/5] Batch 23/938               Loss D: -15.2322, loss G: 2.9749
Epoch [0/5] Batch 24/938               Loss D: -15.4461, loss G: 3.0708
Epoch [0/5] Batch 25/938               Loss D: -15.7034, loss G: 3.1695
Epoch [0/5] Batch 26/938               Loss D: -15.8655, loss G: 3.2650
Epoch [0/5] Batch 27/938               Loss D: -16.0800, loss G: 3.3576
Epoch [0/5] Batch 28/938               Loss D: -16.2794, loss G: 3.4508
Epoch [0/5] Batch 29/938               Loss D: -16.4956, loss G: 3.5395
Epoch [0/5] Batch 30/938               Loss D: -16.7489, loss G: 3.6349
Epoch [0/5] Batch 31/938               Loss D: -16.9096, loss G: 3.7230

As you can see, the loss of the generator loss G seems to be increasing over batches. Despite this, I can definitely see improvements over the images generation using tensorboard:

batch_0:

enter image description here

batch_31:

enter image description here

So it's not too clear to me why, given that the generator loss is increasing, we obtain better generated results.

Also, is it reasonable to obtain negative values for the losses of discriminator and generator?

$\endgroup$

0

You must log in to answer this question.

Browse other questions tagged .