Say I have a mini-batch of size 32, and I have 10 such batches. Assuming I only run it for one epoch (just for the sake of understanding it), Will the weights be updated using the gradients of one mini-batch, or will it be done after all the 10 mini batches have passed through?

Intuitively for me, it ought to be the first one because otherwise, the only difference between Batch-GD and mini-batch GD will be the size of the batch.

  • 1
    $\begingroup$ In one epoch, weights will be updated 10 times for the above case. $\endgroup$ Oct 17 at 13:18
  • $\begingroup$ Thanks, such a silly thing but it was bothering a lot. $\endgroup$ Oct 17 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.