1
$\begingroup$

Why do the GAN's loss functions use an expectation (sum + division) instead of a simple sum?

$\endgroup$
0

1 Answer 1

0
$\begingroup$

I'm assuming you mean the original loss explained in the original GAN paper. This practice was by design and for interpretation. An expectation is what your expecting to get, which is generally a good objective when your are sampling. Note this isn't just in GANs, it's in most objective functions used in a widespread of problems.

In general practice, though, dividing is good because it works as a normalization. Let's say you train one batch with 10 elements and the next with 8. Wouldn't you want each training example to be weighted equally?

Also, even if you use equal-sized batches, then the "division" you mention gets eaten up by the learning rate if you're using some form of gradient scheme.

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .