0
$\begingroup$

When we train the GAN we usually train the discriminator first then the generator, first we stop the generator from updating its weight by removing it from the computation graph, using fake_image.detach()

noise=get_noise(num_images,z_dim,device=device)
fake_images=gen(noise)

disc_fake_pred = disc(fake_images.detach()) 
disc_fake_loss=criterion(disc_fake_pred,torch.zeros_like(disc_fake_pred)) 
disc_real_pred = disc(real)
disc_real_loss=criterion(disc_real_pred,torch.ones_like(disc_real_pred))
    
disc_loss = (disc_real_loss + disc_fake_loss)/2
disc_loss.backward()
disc_opt.step()

But when we want to train the discriminator we don't stop discriminator weights all we do is just leave the generator weights in the computational graph, this means discriminator and generator weights will be updated since we didn't stop it

gen_opt.zero_grad()
z_noise = get_noise(num_images, z_dim, device)
fake_images = gen(z_noise)
disc_fake_pred = disc(fake_images) 
gen_loss = criterion(disc_fake_pred, torch.ones_like(disc_fake_pred)) 
gen_loss.backward()
gen_opt.step()

But in

$\endgroup$

1 Answer 1

1
$\begingroup$

Since you are using Pytorch, an optimizer can only update parameters wrapped inside it since initializing.

Therefore, even if the gradient of discriminator parameters exists while updating generator, discriminator parameters are still stable if you don't call disc_opt.step().

$\endgroup$
1
  • $\begingroup$ you are correct $\endgroup$ Aug 16, 2022 at 18:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .