Questions tagged [derivative]
The derivative tag has no usage guidance.
10
questions with no upvoted or accepted answers
2
votes
0
answers
63
views
What is the dimensionality of these derivatives in the paper "Active Learning for Reward Estimation in Inverse Reinforcement Learning"?
I'm trying to implement in code part of the following paper: Active Learning for Reward Estimation in Inverse Reinforcement Learning.
I'm specifically referring to section 2.3 of the paper.
Let's ...
- 141
1
vote
1
answer
80
views
What does it mean "having Lipschitz continuous derivatives"?
We can enforce some constraints on functions used in deep learning in order to guarantee optimizations. You can find it in Numerical Computation of the deep learning book.
In the context of deep ...
- 3,201
1
vote
0
answers
72
views
BlackOut - ICLR 2016: need help understanding the cost function derivative
In the ICLR 2016 paper BlackOut: Speeding up Recurrent Neural Network Language Models with very Large Vocabularies, on page 3, for eq. 4:
$$ J_{ml}^s(\theta) = log \ p_{\theta}(w_i | s) $$
They have ...
- 151
0
votes
0
answers
27
views
Why Is There The Term 1/m In Backpropagation
In backpropagation the gradients are used to update the weights using the formula
$$w = w - \alpha \frac{dL}{dw}$$
and the loss gradient w.r.t. weights is
$$\frac{dL}{dw} = \frac{dL}{dz} \frac{dz}{dw} ...
- 11
0
votes
1
answer
58
views
What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?
I have a question about the Grad-CAM++ paper. I do not understand how the following equation (10) for the alphas is obtained:
$$
\alpha_{ij}^{kc} =
\frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}}
{...
- 131
0
votes
0
answers
28
views
How to calculate the gradient (or derivative) of y = f(x) of y w.r.t x where y represents the order statistics divided by median of x?
How to calculate the gradient (or derivative) of y = f(x) of y w.r.t x where y represents the order statistics divided by median of x?
For instance x is ...
- 101
0
votes
1
answer
58
views
How the vector-space isomorphism between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$ guarantees reshaping matrices to vectors?
Consider the following paragraph from section 5.4 Gradients fo Matrices of the chapter Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.
Since ...
- 3,201
0
votes
1
answer
86
views
What is the rigorous and formal definition for the direction pointed by a gradient?
Consider the following definition of derivative from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.
Definition 5.2 (...
- 3,201
0
votes
1
answer
39
views
How to understand slope of a (non-convex) function at a point in domain?
Consider the following paragraph from Numerical Computation of deep learning book that says derivative as a slope of the function curve at a point
Suppose we have a function $y= f(x)$, where both $x$ ...
- 3,201
0
votes
0
answers
137
views
Derivation of regularized cost function w.r.t activation and bias
In regularzied cost function a L2 regularization cost has been added.
Here we have already calculated cross entropy cost w.r.t $A, W$.
As mentioned in the regularization notebook (see below) in ...
- 151