New answers tagged

1

Let's look at the definition of gradient: In vector calculus, the gradient of a scalar-valued differentiable function $f$ of several variables is the vector field (or vector-valued function) $\nabla f$ whose value at a point $p$ is the vector $r^{[a]}$ whose components are the partial derivatives of $f$ at $p .^{[1][2][3][4][5][6][7][8][9]}$ That is, for $f:...


1

Does the popular packages like PyTorch, Tensorflow, Keras, etc., use this or a variant of this algorithm to find the gradients at a particular point? Yes. This is effectively what back-propagation is. However, there are a couple of important details: Using a loss function flattens the matrix form you have to a vector, because with a loss function the ...


Top 50 recent answers are included