Timeline for What is the right way to train a generator in a GAN?
Current License: CC BY-SA 4.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 16, 2020 at 12:59 | comment | added | Neil Slater | @AntonYellow: You need the discriminator included in order to have a measurable loss value for the generator's output, plus the gradients associated with that loss. The generator cannot compare to the goal of creating a realistic fake by itself - it outputs an image, and has no way to get a score for that image. If this was not a GAN, and if you expected a specific/precise image then you could get a score, e.g. if you knew what exact image a generator should make given the input, then you could measure the difference and get a gradient. But with a GAN you cannot, the image is random. | |
| Oct 16, 2020 at 12:38 | comment | added | AntonYellow | Ok good for the Nash eq. Get back on what we were, the last two bullet points in your answer mean that we run the minibatch and get the loss respect the overall cost function (gen+discriminator) and then backpropagate the signal towards the generator, but the discriminator is not touched by this update, right? If it is the case, why use a combined model and don't stick only with the separated generator. I am near the answer but I need a further push:-) to understand why this interconnection is needed | |
| Oct 16, 2020 at 12:14 | comment | added | AntonYellow | [Run the minibatch forward to get loss and backpropagate to get gradients for the whole network including the generator. Apply a gradient step (usually via some optimiser, such as Adam).] | |
| Oct 16, 2020 at 10:38 | comment | added | Neil Slater | @AntonYellow Yes then you go back to (1) in your description and repeat. I don't think it is really a Nash equilibrium - although there is a game-like feel to it, the networks are not trying to optimise decisions or actions, each only has one action and is trying to improve their chance of success. The equilibrium point depends critically on how well the training data of real images covers the population of all possible images of the thing you want the generator to learn, plus the limits of sophistication of the dsicriminator | |
| Oct 16, 2020 at 10:32 | history | edited | Neil Slater | CC BY-SA 4.0 |
added 596 characters in body
|
| Oct 16, 2020 at 9:36 | comment | added | AntonYellow | Many thanks for the prompt answer! Said that, how the algorithm works after that? We have the generator trained with this batch and now it is the discriminator turn right? So I use a new generated batch from the generator + real data set, and then inject in the discriminator alone, that does the new update on weights and so on, (it follows then another round of combined gen and discr to train generator with fix discriminator.....) until the nash equilibrium if obtained. Am I right? | |
| Oct 16, 2020 at 8:53 | history | answered | Neil Slater | CC BY-SA 4.0 |