What are your "current parameters" in Minibatch Stochastic Gradient Descent?

Question

I was reading a book on Deep Learning when I came across a line, more like a few words that didn't make apparent sense.

Thus, we will often settle for sampling a random minibatch of examples every time we need to compute the update, a variant called minibatch stochastic gradient descent. In each iteration, we first randomly sample a minibatch B consisting of a fixed number of training examples. We then compute the derivative (gradient) of the average loss on the minibatch with regard to the model parameters. Finally, we multiply the gradient by a predetermined positive value η and subtract the resulting term from the current parameter values.

We can express the update mathematically as follows ($∂$ denotes the partial derivative):

$(w,b) \leftarrow (w,b) - {η\over |B|} {\Large \Sigma}_{I \in B} \partial_{(w,b)}l^{(i)}(w,b).$

The set cardinality $|B|$ represents the number of examples in each minibatch (the batch size) and $η$ denotes the learning rate.

What does "current" parameter values mean in this context?

You can find the book here -

https://d2l.ai $\rightarrow$ Chapter Linear Neural Networks, Part 3.1.1.4

Answer 1

The current parameter values are the values of the weights $w$ and the biases $b$ in the neurons in each layer. What you are calculating is the gradient of the average loss of the batch relative to the weight and the bias. Since the gradient points "up", or toward greater loss, you multiply by the learning rate and subtract it from the current values.