Why do GAN loss functions use expectation(sum + division) instead of a simple sum?


I'm assuming you mean the original loss explained in The original gan paper. This practice was by design and for interpretation. An expectation is what your expecting to get, which is generally a good objective when your are sampling. Note this isnt just in GAN's its in most objective functions used in a wide spread of problems.

In general practice though, dividing is good because it works as a normalization. Lets say you train one batch with 10 elements and the next with 8... wouldnt you want each training example to be weighted equally?

Also even if you use equal sized batches, then the "division" you mention gets eaten up by the learning rate if your using some form of gradient scheme.

Hope this helped

  • $\begingroup$ Thanks for the explanation, I appreciate it! $\endgroup$ – NNLearner Jun 10 '19 at 1:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.