When training GANs, I can do this:
pseudo code
opt_g = Optimizer(G.params)
opt_d = Optimizer(D.params)
fake1 = G(z1)
l = loss(D(fake1)
l.backward()
opt_g.step()
fake2 = G(z2).detach()
l = loss(-D(fake2)) + loss(D(real))
l.backward()
opt_d.step()
However, I am wondering, if I can reuse G(z1)
or even D(G(z1))
fake = G(z1)
fake_pred = D(fake)
l = loss(fake_pred)
l.backward()
opt_g.step()
# variant 1 (shared fake)
l = D(fake.detach()) + loss(D(real))
# variant 2 (shared prediction)
l = loss(-fake_pred) + loss(D(real))
l.backward()
opt_d.step()
If this was possible, I wonder why it was implemented differently in StyleMapGAN?