1
$\begingroup$

I've read that the discriminator $D$ validates an image $D(x)$, where $x$ is either a real image or a fake one created by the generator, i.e. $ D(G(x))$.

What does the function of the discriminator return? Is it either 0 (marked as fake) or 1 (discriminator thinks the image is real)? I have read that this function returns the whole $\mathbb{R}$, but I don't understand what the output then means.

$\endgroup$
4
  • $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Jun 6, 2022 at 22:28
  • 1
    $\begingroup$ In general, the discriminator returns probability. But, there are some GANs in which the discriminator returns over a range of real numbers. So, both can be true and it depends on the GAN you are considering. For applications, discriminator is used as a binary classifier also; $\endgroup$
    – hanugm
    Jun 7, 2022 at 2:13
  • 2
    $\begingroup$ @hanugm That looks like an answer to me, even if it's short - some correct answers are just short. Maybe just provide some references or names of GANs that return one over the other. $\endgroup$
    – nbro
    Jun 7, 2022 at 20:21
  • $\begingroup$ Notice this is a classification problem, just like any other neural network classification problem. You could think about how a neural network tells us whether an image is a cow or a sheep. $\endgroup$
    – user253751
    Jun 9, 2022 at 13:44

2 Answers 2

3
$\begingroup$

Formally, for an input $x$, $D(x)$ gives you the probability of $x$ being real. In this sense $D:\mathcal{X}\rightarrow [0,1]$, where $\mathcal{X}$ is the input space.

That said, the output of the discriminator is a probability (hence within 0 and 1), and you get the prediction (fake or real) by considering the most probable outcome. Informally $D(x) = fake$ for $D(x) < 0.5$ and real otherwise.

@edit: on the issue of Least Squares GAN.

In the first 2 paragraphs of my answer, I considered the case of the original GAN proposed by Goodfellow et al. [1]. Nonetheless other types of GANs exist, that do not employ a sigmoid activation at the output layer of the discriminator. That is the case of the Least Squares GAN of Mao et al. [2], upon which the authors of CycleGAN based themselves on [3].

The authors of [2] raise the issue of using a sigmoid activation in section 3.2:

when updating the generator, this loss function (cross-entropy on sigmoid activations) will cause the problem of vanishing gradients for the samples that are on the correct side of the decision boundary, but are still far from the real data.

The LS-GAN proposes the following workaround: $D(x) \in \mathbb{R}$, thus no sigmoid activation at the end. The loss is then substituted by the least squares loss. As follows, there is no clear encoding for fake and real labels anymore. For that reason the authors of [2] introduce constants $a$ and $b$, such that if $D(x) \approx a \implies x$ is fake, and $D(x) \approx b \implies x$ is real. For the generator, there is another constant $c$ such that $D(G(z)) \approx c \implies G(z)$ is fake. In [3] the authors picked $a = 0$, $b = 1$ and $c = 1$.

References

[1] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.

[2] Mao, X., Li, Q., Xie, H., Lau, R. Y., Wang, Z., & Paul Smolley, S. (2017). Least squares generative adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2794-2802).

[3] Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2223-2232).

$\endgroup$
4
  • $\begingroup$ After further readings, I wanted to ask one more thing. Using the Sigmoid Activation function in the last Layer of the PatchGAN does push the values of the discriminator function between 0-1. Without it whole ℝ is possible as output for D(x). In the original CycleGAN Paper they removed the Sigmoid Activation function and don’t use any other activation function. So without it I don’t understand how does the adversarial loss e.g the expression (D(x) - 1) then makes any sense? If D(x) is a number in ℝ, how does reducing it by one gives me information about the real or fakeness of the image? $\endgroup$ Jun 7, 2022 at 7:49
  • $\begingroup$ As the authors mention in Section 4 of their paper, this is a trick for acquiring better performance. I will edit my answer to comment further on this choice. $\endgroup$ Jun 7, 2022 at 7:55
  • $\begingroup$ Thank you for your detailed answer! In the meantime i stumbeld upon another question regarding the adversarial loss of a CycleGAN. If I understood that correctly a LSGAN uses L2 loss or MSE loss. MSE has additionally the term 1/N. Is that correctly? The LSGAN Paper [2] has 1/2 in the formula in paragraph 3.2. But in the original Paper of the CycleGan [3] the same formula in Chapter 4. does not have the 1/2 upfront. Now to my Question. Where does the 1/2 come from and why is it in the CycleGAN Paper missing? Again thank you very much so far for all the help! $\endgroup$ Jun 7, 2022 at 14:33
  • $\begingroup$ The 1/N comes from the expectation operator, as you approximate it empirically (N being the number of samples taken from a given distribution). Concerning the 1/2, it is actually entirely optional as it does not change the optimal point of a function. $\endgroup$ Jun 7, 2022 at 15:14
0
$\begingroup$

Let me try to explain this way, comment if you think it's incorrect.

Assume a simple linear function, $y=f(x)=ax+b$ where $a \in \mathbb{R}^*$ and $b\in \mathbb{R}$, each value of $y$ is unique, which means we can only get the same $y$ from the same $x$.

In GANs, the Discriminator plays the same role as $f$, with the more complex function and the high dimension input (for example, an MNIST's image will have $32\times32=1024$ dimension), $y$ is no longer unique but still keeps its property, the closer $x$ (same domain), the closer $y$ value.

Post-process the output of the discriminator is to adapt to the loss function, the original GANs of Ian Goodfellow limited the output by a sigmoid function to wrap it in a probability range to input it to a logarithm as cross-entropy. WGAN measures the Wasserstein distance by the real value from the discriminator or Hinge loss clamps the output cap as 0.

If you feel hard with the definition of the domain, take a look at the famous problem "dogs vs cats", it's the binary classification, and the task is to build a deep learning model to distinguish images. There are many types of dogs but they still have common characteristics that make them classified as a "dog" domain. In GANs, the task of the discriminator is the same, separate real and fake domains just like dog and cat.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .