0
$\begingroup$

I've been trying to implement my own neural network library and have been wondering if:

  • The SSE loss function includes the summation of the errors in the other training examples of the mini-batch (each training example's loss in the mini-batch is summed for one big loss)

  • The MSE loss function averages the loss of each individual training example in the mini-batch, and then all those losses are averaged based on the mini-batch size.

or if the summing or averaging of the weight gradients has nothing to do with the loss function used?

I feel like the answer would be clear if I knew other loss functions, if it did matter that would mean the weight gradients should be averaged for MSE and summed for SSE in mini-batch gradient descent?

Improve this question
$\endgroup$
2
  • $\begingroup$ seems at least a partial dup of this. let me know. $\endgroup$
    – nbro
    Commented Jan 28, 2022 at 15:51
  • $\begingroup$ hi, I've seen that one before. I don't think it answers my question. $\endgroup$
    – jake_prentice
    Commented Jan 28, 2022 at 16:03

1 Answer 1

Reset to default
1
$\begingroup$

What you stated looks correct :-

The SSE loss function includes the summation of the errors in the other training examples of the mini-batch (each training example's loss in the mini-batch is summed for one big loss) SSE = Summation of (y-y')squared for all samples in mini-batch

The MSE loss function averages the loss of each individual training example in the mini-batch, and then all those losses are averaged based on the mini-batch size. MSE = (1/m)SSE

Improve this answer
$\endgroup$
1
  • $\begingroup$ You can format the formulas and math symbols in this answer with latex. I would recommend that you do that. $\endgroup$
    – nbro
    Commented Jan 31, 2022 at 12:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .