Per google machine-learning glossary, when I have 100 training examples and update my model for each training example, if I train my model 5 epochs without early-stop, there are 500 iterations in total, is my understanding correct?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Updating model for each training example means batch size of 1, aka stochastic gradient descent(SGD).
1 iteration is defined as forward propagate, calculate loss, backpropagate and finally update weights.
Since batch size is 1, running 5 epochs on 100 training examples with SGD means you will do 500 iterations, yes.
