I am looking for literature recommendations regarding GANs with multiple discriminators. In particular, I am looking for examples where each discriminator has a slightly different learning objective, rather than learning on different data. My thinking was that the generator sometimes is exposed to reward sparsity: i.e. its samples get constantly rejected. Having multiple objectives be optimised through multiple discriminators might help alleviate this problem to a certain extent, as it increases the chance of positive feedback from one of the discriminators. Do you know of any examples, and does GAN training with multiple discriminators generally make sense or does it make training more unstable for some reason I have not considered?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
As Sadaf pointed out, MD-GAN is a well-known one. But it doesn't have multiple objectives as you wanted.
One GAN architecture that has 2 discriminators, where both have different objectives, is FairGAN, which tries to increase synthetic data fairness by having an additional discriminator which tries to see if the data is fair (supervised on some feature of the data). I personally love the paper, and I think it is along the lines of what you are looking for.
For your question, "Does GAN training with multiple discriminators generally make sense?" Having multiple objectives is not necessarily sensical. For FairGAN, they have 2 objectives, namely 'generating realistic data' and 'generating fair data'. Hence, 2 discriminators makes sense. However, this only works if you have particular data characteristics, such that you can train a second discriminator on that data (because you often need labels). I think that having multiple objectives is only possible in specific situations where you have data which suits such a multi-objective solution. In general, generative models are self-supervised and hence super easily applicable. This is not the case for multi-objective stuff, as the added objective probably needs some data-characteristic.
In a different light, ADS-GAN has the additional objective of generating private data, and does so by adding an additional loss component which can be computed directly using the generated data. This is not an additional discriminator but is an additional objective.
Seeing you specifically mentioned 'rather than learning on different data', I'm assuming you know about PATE-GAN, which employs multiple discriminators to make the generative model have differential privacy by having each teacher discriminator train on a disjoint set of data. (I simply added this for others seeing this later).
answered Dec 16, 2022 at 10:35
-
$\begingroup$ Thank you for your thorough reply. Thinking about it, what I actually want to do is pre-train a second generator, D2, on another objective and use it later to give the generator feedback, but freeze the weights of D2. So G and D will alternately learn like usual, but D2 is only there to provide more feedback to G, not to learn by itself. I have two objectives that share a common goal but have a different dificulty level. Do you know of instances where this has been done? $\endgroup$ Commented Dec 16, 2022 at 13:08
-
1$\begingroup$ Could you check if you used the terms discriminator and generator correctly everywhere? You say you want to pretrain a generator D2? Im assuming you mean a discriminator? If you pretrain it, (and not during the rest of training), its basically operating as a loss function, so i'd refer you to the ADS-GAN paper i linked. $\endgroup$ Commented Dec 16, 2022 at 13:25
-
$\begingroup$ I meant a second discriminator, my apologies. Once I noticed the mistake I could no longer edit my comment. And thank you, I will have a look at the ADS-GAN paper. $\endgroup$ Commented Dec 16, 2022 at 13:38
$\begingroup$
$\endgroup$
MD-GAN (multi-Discriminator Generative Adversarial Networks for Distributed Datasets ) would be among the ones that you might be looking for. It has been proposed a while ago now.
This has been proposed so that we can utilize high computation over distributed computing.
Most of the work has been done around multiple generators than on discriminators like Mc-GAN,S-GAN,Mg-GAN etc.
The probable reason is, that Discriminators are usually doing good, as their job is mostly classification while Generation is the hard problem, hence most research is around that only
