2
$\begingroup$

I am quite new to GAN and I am reading about WGAN vs DCGAN.

Relating to the Wasserstein GAN (WGAN), I read here

Instead of using a discriminator to classify or predict the probability of generated images as being real or fake, the WGAN changes or replaces the discriminator model with a critic that scores the realness or fakeness of a given image.

In practice, I don't understand what the difference is between a score of the realness or fakeness of a given image and a probability that the generated images are real or fake.

Aren't scores probabilities?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ I didn't read that paper and I don't have time now to read it, but the term score in statistics (and machine learning) sometimes refers to a specific concept. Take a look at this. If nobody provides an answer, you may want to read the WGAN paper (to know how they defined the score there) and the linked Wikipedia article, and then provide a formal answer to your own question below ;) $\endgroup$
    – nbro
    Commented Jan 25, 2021 at 13:10

1 Answer 1

Reset to default
1
$\begingroup$

Figure 3 in the original WGAN paper is actually quite helpful to understand the difference between the score in WGAN and the probability in GAN (see screenshot below). The blue distribution are real samples, and the green one are fake samples. The Vanilla GAN trained in this example identifies the real samples as '100% real' (red curve) and the fake samples as '100% fake'. This leads to the problem of vanishing gradients and the well-known mode collapse of original GANs.

The Wasserstein GAN, on the other hand, gives each sample a score. The benefit of the score is that we can now identify samples that are more likely real than others, or more likely fake. For example, the further to the left a distribution is located, the more negative the WGAN score will be. We have therefore a continuum that doesn't end in 0 and 1, but can compare between samples that are 'good' and those which are 'better'. A normal GAN would identify both as 'good', making further improvement difficult.

Figure 3 from Arjovsky et al, 2017

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks! There is an aspect which is not clear to me, if a binary cross entropy is used as activation for the last layer of the discriminator you will obtain for each sample a probability for belonging to a class. So "the further to the left a distribution is located" the lower will be the probability for the samples to belong to the fake class. The only problem that could come to mind is that the classifier is so sure of the choice, since in this case the samples are even linearly separable, which will always return probability about 1 for one of the two classes, is it correct? $\endgroup$
    – Stefano Barone
    Commented Jan 27, 2021 at 10:06
  • 1
    $\begingroup$ Yes! That's at least one of the reasons that lead to mode collapse in traditional GANs, i.e. the inability to recover a multi-modal distribution (see example in Figure 2 of the WGAN paper). If the generator outputs one of the modes, but the samples are already judged as "very real looking" (that is, "very far on the left"), there is no motivation for it to improve as the gradients are vanishing and don't give useful information. If instead the Wasserstein distance is used, which can be identified as a score, we don't have this problem but a monotonous curve with useful gradients. $\endgroup$
    – Mafu
    Commented Jan 28, 2021 at 0:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .