Timeline for Does average loss function in GAN training is just an approximation of value function and does not ensure convergence of generator and discriminator?

Current License: CC BY-SA 4.0

7 events

when toggle format	what		by	license	comment
Aug 3, 2021 at 8:45	comment	added	Aray Karjauv		@hanugm I don't think the mean is related to the probabilities of the data in this case. It has more to do with the importance of the gradient in the batch. The gradient for all data points is equally important. You can update your model using one data point at a time, so you get rid of the mean. The likelihood of the data is encoded in your dataset. If your dataset is unbalanced, you can sample a specific class with a certain probability, for example, using WeightedRandomSampler in PyTorch.
Aug 3, 2021 at 2:19	comment	added	hanugm		yeah, I understood. But I am not sure whether considering our own probability is theoretically correct or nor. So, I am still in confusion.
Aug 3, 2021 at 1:15	comment	added	Aray Karjauv		I guess I answered the question about $m$ - we assume each image has the same probability (we can also sample images with certain probabilities). This is not so important. We could have made a batch size 1, but as far as I know, batch optimization is less prone to getting stuck in local minima. As for the noise, it is only used to generate images, so we can replace $G(z)$ with $\hat{x} \sim P_g$
Aug 2, 2021 at 23:09	comment	added	hanugm		My doubt is in fixing up the probabilities of noise, generated and real samples as $\dfrac{1}{m}$, where $m$ is just a hyperparameter we are selecting and does not depends on actual probability. So, how can we take that?
Aug 2, 2021 at 12:39	history	edited	Aray Karjauv	CC BY-SA 4.0	added 146 characters in body
Aug 2, 2021 at 12:32	history	edited	Aray Karjauv	CC BY-SA 4.0	added 146 characters in body
Aug 2, 2021 at 12:24	history	answered	Aray Karjauv	CC BY-SA 4.0