Questions tagged [wasserstein-metric]
For questions about the Wasserstein metric/distance (or Kantorovich–Rubinstein metric), which is a metric (distance function) defined between probability distributions.
7 questions
1
vote
0
answers
62
views
How exactly do you backpropagate the gradient penalty in WGAN-GP?
I am trying to implement WGANs from scratch.
The loss function for the critic is given by :
which i implement in my code as L = average(real output) - average(fake output) + lambda*GP.
For ...
2
votes
1
answer
485
views
Why do we use a linear interpolation of fake and real data to penalize the gradient of discriminator in WGAN-GP
I'm trying to better frame/summarize the formulations and motivations behind Wasserstein GAN with gradient penalty, based on my understanding.
For the basic GAN we are trying to optimize the following ...
- 307
8
votes
2
answers
4k
views
Why is KL divergence used so often in Machine Learning?
The KL Divergence is quite easy to compute in closed form for simple distributions -such as Gaussians- but has some not-very-nice properties. For example, it is not symmetrical (thus it is not a ...
- 253
3
votes
1
answer
698
views
Are there some notions of distance between two policies?
I want to determine some distance between two policies $\pi_1 (a \mid s)$ and $\pi_2 (a \mid s)$, i.e. something like $\vert \vert \pi_1 (a \mid s) - \pi_2(a \mid s) \vert \vert$, where $\pi_i (a\mid ...
- 297
1
vote
0
answers
200
views
WGAN-GP Loss formalization
I have to write the formalization of the loss function of my network, built following the WGAN-GP model. The discriminator takes 3 consecutive images as input (such as 3 consecutive frames of a video) ...
- 111
4
votes
1
answer
607
views
What is the reason for mode collapse in GAN as opposed to WGAN?
In this article I am reading:
$D_{KL}$ gives us inifity when two distributions are disjoint. The value of $D_{JS}$ has sudden jump, not differentiable at $\theta=0$. Only Wasserstein metric provides ...
- 141
1
vote
0
answers
43
views
Under what conditions can one find the optimal critic in WGAN?
The Kantorovich-Rubinstein duality for the optimal transport problem implies that the Wasserstein distance between two distributions $\mu_1$ and $\mu_2$ can be computed as (equation 2 in section 3 in ...
- 51