When we train the GAN we usually train the discriminator first then the generator, first we stop the generator from updating its weight by removing it from the computation graph, using fake_image.detach()
noise=get_noise(num_images,z_dim,device=device)
fake_images=gen(noise)
disc_fake_pred = disc(fake_images.detach())
disc_fake_loss=criterion(disc_fake_pred,torch.zeros_like(disc_fake_pred))
disc_real_pred = disc(real)
disc_real_loss=criterion(disc_real_pred,torch.ones_like(disc_real_pred))
disc_loss = (disc_real_loss + disc_fake_loss)/2
disc_loss.backward()
disc_opt.step()
But when we want to train the discriminator we don't stop discriminator weights all we do is just leave the generator weights in the computational graph, this means discriminator and generator weights will be updated since we didn't stop it
gen_opt.zero_grad()
z_noise = get_noise(num_images, z_dim, device)
fake_images = gen(z_noise)
disc_fake_pred = disc(fake_images)
gen_loss = criterion(disc_fake_pred, torch.ones_like(disc_fake_pred))
gen_loss.backward()
gen_opt.step()
But in