# How does the output distribution of a GAN change if the parameters are slightly purturbed?

Suppose $$G_{\phi}:\mathcal{Z}\rightarrow \mathcal{X}$$ is a generator (neural network, non-invertible) that can sample from some distribution $$\pi$$ on $$\mathcal{X}$$. That is, $$G_{\phi}(z)\sim \pi$$ when $$z\sim \mathcal{N}(0,I)$$. Let $$\phi+\delta_{\phi}$$ represent a (small) perturbation of the parameters of $$G_{\phi}$$ and let $$G_{\phi+\delta_{\phi}}(z)\sim \pi'$$ when $$z\sim \mathcal{N}(0,I)$$.

Are there any results that quantify or bound $$\mathcal{D}(\pi,\pi')$$ in terms of $$\delta_{\phi}$$, where $$\mathcal{D}$$ is a distance measure for distributions (let's say KL-divergence, or the Wasserstein-1 distance)?

Basically, I want to know what kind of geometry is induced on the space of distributions by the Euclidian geometry on the parameter space of a generative adversarial network.

To explain further, let's consider a parametric family of distributions $$p_{\phi}$$, where $$\phi\in\Phi$$ (some parameter space). It is a fairly well-known result in statistics that $$\text{KL}(p_{\phi}||p_{\phi+\delta_{\phi}})\approx \frac{1}{2}\delta_{\phi}^\top F_{\phi} \delta_{\phi}$$, where $$F_{\phi}$$ is the Fisher information matrix. When the family $$p_{\phi}$$ is generated by a GAN with parameter $$\phi$$ (in which case we don't know $$p_{\phi}$$ in closed-form), can we have an analogous result?

• I'm not really sure if I understood your question correctly, but here is some ideas. Let's say we have a 32x32 RGB image. It gives us $32^2\cdot 256^3$ dimensional space. But it turns out, that we can encode an image using latent features. When we talk about a distribution space in terms of GAN, we mean a lower-dimensional latent features manifold embedded in that higher-dimensional space. StyleGAN2 has done excellent research on this topic. Apr 2 '21 at 18:11
• You can also take a look at this paper. The authors propose 50% dropdown layers that are used to generate images with some noise to make the model explore the underlaying manifold. Apr 2 '21 at 18:18
• Please see the clarification I addded. Hope that clarifies my question. Apr 2 '21 at 19:05
• What do you mean by generating a probability distribution $p_{\phi}$ by a GAN? The GAN itself (namely the generator) is a probability distribution from which we can sample data (images). Apr 2 '21 at 22:04