Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
I've read that the discriminator $D$ validates an image $D(x)$, where $x$ is either a real image or a fake one created by the generator, i.e. $ D(G(x))$.
What does the function of the discriminator return? Is it either 0 (marked as fake) or 1 (discriminator thinks the image is real)? I have read that this function returns the whole $\mathbb{R}$, but I don't understand what the output then means.
Formally, for an input $x$, $D(x)$ gives you the probability of $x$ being real. In this sense $D:\mathcal{X}\rightarrow [0,1]$, where $\mathcal{X}$ is the input space.
That said, the output of the discriminator is a probability (hence within 0 and 1), and you get the prediction (fake or real) by considering the most probable outcome. Informally $D(x) = fake$ for $D(x) < 0.5$ and real otherwise.
@edit: on the issue of Least Squares GAN.
In the first 2 paragraphs of my answer, I considered the case of the original GAN proposed by Goodfellow et al. [1]. Nonetheless other types of GANs exist, that do not employ a sigmoid activation at the output layer of the discriminator. That is the case of the Least Squares GAN of Mao et al. [2], upon which the authors of CycleGAN based themselves on [3].
The authors of [2] raise the issue of using a sigmoid activation in section 3.2:
when updating the generator, this loss function (cross-entropy on sigmoid activations) will cause the problem of vanishing gradients for the samples that are on the correct side of the decision boundary, but are still far from the real data.
The LS-GAN proposes the following workaround: $D(x) \in \mathbb{R}$, thus no sigmoid activation at the end. The loss is then substituted by the least squares loss. As follows, there is no clear encoding for fake and real labels anymore. For that reason the authors of [2] introduce constants $a$ and $b$, such that if $D(x) \approx a \implies x$ is fake, and $D(x) \approx b \implies x$ is real. For the generator, there is another constant $c$ such that $D(G(z)) \approx c \implies G(z)$ is fake. In [3] the authors picked $a = 0$, $b = 1$ and $c = 1$.
[1] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.
[2] Mao, X., Li, Q., Xie, H., Lau, R. Y., Wang, Z., & Paul Smolley, S. (2017). Least squares generative adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2794-2802).
[3] Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2223-2232).
Let me try to explain this way, comment if you think it's incorrect.
Assume a simple linear function, $y=f(x)=ax+b$ where $a \in \mathbb{R}^*$ and $b\in \mathbb{R}$, each value of $y$ is unique, which means we can only get the same $y$ from the same $x$.
In GANs, the Discriminator plays the same role as $f$, with the more complex function and the high dimension input (for example, an MNIST's image will have $32\times32=1024$ dimension), $y$ is no longer unique but still keeps its property, the closer $x$ (same domain), the closer $y$ value.
Post-process the output of the discriminator is to adapt to the loss function, the original GANs of Ian Goodfellow limited the output by a sigmoid function to wrap it in a probability range to input it to a logarithm as cross-entropy. WGAN measures the Wasserstein distance by the real value from the discriminator or Hinge loss clamps the output cap as 0.
If you feel hard with the definition of the domain, take a look at the famous problem "dogs vs cats", it's the binary classification, and the task is to build a deep learning model to distinguish images. There are many types of dogs but they still have common characteristics that make them classified as a "dog" domain. In GANs, the task of the discriminator is the same, separate real and fake domains just like dog and cat.
