I have implemented DCGAN's myself and have been studying GAN's for over a month now. Now I am implementing the pggans but I encountered a sentence
When we measure the distance between the training distribution and the generated distribution, the gradients can point to more or less random directions if the distributions do not have substantial overlap (https://arxiv.org/pdf/1710.10196.pdf)
but we do never compare the distribution between training and generated distributions in gans a far I know when we train the gan
fixed_noise = to.randn(num_test_samples, 100).view(-1,100, 1, 1)
for epoch in range(opt.number_epochs):
D_losses = []
G_losses = []
for i,(images, labels) in enumerate(dataloader):
minibatch = images.size()[0]
real_images = Variable(images.cuda())
real_labels = Variable(to.ones(minibatch).cuda())
fake_labels = Variable(to.zeros(minibatch).cuda())
##Train discriminator
#First with real data
D_real_Decision = discriminator(real_images).squeeze()
D_real_loss = criterion(D_real_Decision,real_labels)
#with fake data
z_ = to.randn(minibatch,100 ).view(-1, 100, 1, 1)
z_ = Variable(z_.cuda())
gen_images = generator(z_)
D_fake_decision = discriminator(gen_images).squeeze()
D_fake_loss = criterion(D_fake_decision,fake_labels)
## back propagation
D_loss = D_real_loss + D_fake_loss
discriminator.zero_grad()
D_loss.backward()
opt_Disc.step()
# train generator
z_ = to.randn(minibatch,100 ).view(-1, 100, 1, 1)
z_ = Variable(z_.cuda())
gen_images = generator(z_)
D_fake_decisions = discriminator(gen_images).squeeze()
G_loss = criterion(D_fake_decisions,real_labels)
discriminator.zero_grad()
generator.zero_grad()
G_loss.backward()
opt_Gen.step()
we just train the discriminator on real and fake images, and then train generator on the outputs of discriminator on generated images,
So Please let me know where do we compare the distribution between training and generated distribution, and how do generator learns to mimic the training samples