In the stochastic gradient descent algorithm, the weight update happens for every training sample.
In the mini-batch gradient descent algorithm, the weight update happens for every batch of training samples.
In the batch gradient descent algorithm, the weight update happens for all samples in the training dataset.
I am confused with the procedure of training that happens in the mini-batch gradient descent algorithm. I am guessing one of the following two must be correct
Passing each input individually at each layer and calculating the output. This happens for a number of training samples that are equal to batch size.
Passing a batch of inputs at once at each layer and collecting the batch output at each layer.
Which of the above is true in general implementations of mini-batch gradient descent algorithms to train your neural networks?