I am working on a pix2pix GAN model that was inspired by the code in this Github repository. The original code is working and I have already customized most of the code for my needs. However, there is one part I am unable to understand.

The pix2pix GAN is a conditional GAN network that takes an image as a condition and outputs a modified image - such as blurry to clear, facades to buildings, filling up cut out part of an image, etc. The combined model thus takes as input a conditional image, the discriminator compares it with the dummy matrix named valid or fake, containing 0s or 1s according to validity (0 for generated samples, 1 for real samples). The generator loss is according to similarity with real sample + discriminator. The following code corresponds to what I told:

self.combined = Model(inputs=[img_A, img_B], outputs=[valid, fake_A])
self.combined.compile(loss=['mse', 'mae'],
                      loss_weights=[1, 100],

The losses are thus set as MSE for discriminator output and MAE for generator. That seems to be OK, but I can not understand why the implementation uses 1 and 100 for the weights of the discriminator and generator losses, respectively, which seems to imply that the discriminator loss is 100 times lower than the loss of the generator. I couldn't find the reason in the original article. Are my understandings of the GAN incorrect?

Disclaimer: I have posted this question on Stats SE, but have no luck with answers. Maybe it is more suitable for AI.

  • $\begingroup$ Maybe you can take a look at the original/official implementation of pix2pix and see if they are doing something similar: github.com/phillipi/pix2pix. $\endgroup$ – nbro Jan 19 at 11:48

After further research, I have found the answer. nbro was of course right, the weighting is not implementation dependant, it was already introduced in the paper (arXiv). However, there is minimal information about that, only it is mentioned within the optimization function:

$\arg \min_G \max_D \mathcal{L}_{cGAN}(G,D) + \lambda \mathcal{L}_{L1}(G)$

In fact, the parameter lambda stands for weight of $L_1$ loss. There should be a second parameter $\epsilon$ for $\mathcal{L}_{cGAN}(G,D)$ because in real implementation you can actually change that as well. In theory, however, only modulation of $L_1$ loss is needed. The paper does not state why the $\lambda$ parameter is needed, only mentions that setting it to 0 will lead to pure cGAN implementation. David Brownlee in his blog states:

The adversarial loss influences whether the generator model can output images that are plausible in the target domain, whereas the L1 loss regularizes the generator model to output images that are a plausible translation of the source image. As such, the combination of the L1 loss to the adversarial loss is controlled by a new hyperparameter lambda, which is set to 10, e.g. giving 10 times the importance of the L1 loss than the adversarial loss to the generator during training.

The loss weights thus are hyperparameters that tell the network how much plausible translation of the source image do we need.

  • 1
    $\begingroup$ In addition, I would like to point out that I've seen this weighting of one component of the loss also being used in the context of variational auto-encoders (VAEs) and variational Bayesian neural networks, where the loss function is usually the ELBO (Evidence Lower BOund), which is composed of 2 terms, which can be weighted. There's actually a little bit of debate about what the correct weighting should be (at least in the context of variational BNNs). $\endgroup$ – nbro Jan 27 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.