Why do the GAN's loss functions use an expectation (sum + division) instead of a simple sum?


I'm assuming you mean the original loss explained in the original GAN paper. This practice was by design and for interpretation. An expectation is what your expecting to get, which is generally a good objective when your are sampling. Note this isn't just in GANs, it's in most objective functions used in a widespread of problems.

In general practice, though, dividing is good because it works as a normalization. Let's say you train one batch with 10 elements and the next with 8. Wouldn't you want each training example to be weighted equally?

Also, even if you use equal-sized batches, then the "division" you mention gets eaten up by the learning rate if you're using some form of gradient scheme.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.