All Questions
Tagged with calculus machine-learning
6 questions
0
votes
1
answer
38
views
Why is gradient clipping not preventing my gradient descent from going out of bounds?
I'm exploring machine learning and currently studying calculus, specifically gradient descent. To practice, I am using the function: $$ f(x,y)= x^{2}y $$
I have implemented the gradient descent ...
0
votes
2
answers
243
views
How the vector-space isomorphism between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$ guarantees reshaping matrices to vectors?
Consider the following paragraph from section 5.4 Gradients fo Matrices of the chapter Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.
Since ...
- 3,990
0
votes
1
answer
52
views
Which is more popular/common way of representing a gradient in AI community: as a row or column vector?
Consider the following remark about writing gradients from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.
Remark (...
- 3,990
0
votes
1
answer
1k
views
What is the correct formula for updating the weights in a 1-single hidden layer neural network?
I'm creating a neural network with 3 layers and no bias.
On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was:
$$\Delta W_{j,k} = (...
- 111
2
votes
0
answers
321
views
Is Gradient Descent algorithm a part of Calculus of Variations?
As in https://en.wikipedia.org/wiki/Calculus_of_variations
The calculus of variations is a field of mathematical analysis that
uses variations, which are small changes in functions and ...
- 1,318
3
votes
2
answers
2k
views
What is a bad local minimum in machine learning?
What is "bad local minima"?
The following papers all mention this expression.
Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit
limination of All Bad Local ...
- 200