I am trying to understand how the loss function from the original GAN paper

$$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-D(G(\boldsymbol{z})))]$$

relates to generalization in the Nagarajan & Kolter (2017)

$$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[f(D(x))]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[f(-D(G(z)))]$$

where supposedly the original loss function could be recovered via

$$f(x) = -\log(1 + \exp(-x))$$

But when I try to substitute $f$ into the generalized version of the loss function I am not getting the original one. What would be the correct steps to recover the original loss function?


1 Answer 1


There are some definitions that may cause confusion here.

In the original GANs (the first formula), the output from the discriminator connects to a sigmoid activation, the second formula is the real value from the last layer of the D model which you can find in section 3.1

note that this convention slightly differs from the standard formulation in that in this case the discriminator outputs the real-valued “logits”

Rewrite the first formula, consider the output from $D(x)$ is the real value, it will become:

$$ \min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log {\frac{1}{1+exp(-D(x))}})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-\frac{1}{1+exp(-D(G(z)))})] $$

Now, it's able to proof the first and second formula is the same.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .