I am trying to comprehend how the Gradient Descent works.
I understand we have a cost function which is defined in terms of the following parameters,
$J(𝑤_{1},𝑤_{2},.... , w_{n}, b)$
the derivative would tell us which direction to adjust the parameters.
i.e. $\dfrac{dJ(𝑤_{1},𝑤_{2},.... , w_{n}, b)}{d(𝑤_{1}}$ is the rate of change of the cost w.r.t $𝑤$
The lecture kept on saying this very valuable as we are asking the question how should I change $𝑤$ to improve the cost?
But then the Lecturer presented $w_{1}$, $w_{2}$, ... as scaler value, How can we differentiate a scalar value.
I am fundamentally missing what is happening.
Can anyone please guide me to any blog post, a book that I should read to understand better?