Questions tagged [calculus]
For questions related to calculus (developed, among others, by Newton and Leibniz), in the context of AI (and, in particular, machine learning).
9 questions with no upvoted or accepted answers
3
votes
0
answers
233
views
Best calculus books for Deep Learning
Recommend some calculus books for Deep Learning and neural networks. I know what is integration, differentiation, derivates, limits on a based level. I would like to understand on deep level the ...
- 31
2
votes
0
answers
53
views
Is there any wrong in my focal loss derivation?
Assume $\mathbf{X} \in R^{N, C}$ is the input of the softmax $\mathbf{P} \in R^{N, C}$, where $N$ is number of examples and $C$ is number of classes:
$$\mathbf{p}_i = \left[ \frac{e^{x_{ik}}}{\sum_{j=...
- 121
2
votes
0
answers
321
views
Is Gradient Descent algorithm a part of Calculus of Variations?
As in https://en.wikipedia.org/wiki/Calculus_of_variations
The calculus of variations is a field of mathematical analysis that
uses variations, which are small changes in functions and ...
- 1,318
1
vote
0
answers
82
views
BlackOut - ICLR 2016: need help understanding the cost function derivative
In the ICLR 2016 paper BlackOut: Speeding up Recurrent Neural Network Language Models with very Large Vocabularies, on page 3, for eq. 4:
$$ J_{ml}^s(\theta) = log \ p_{\theta}(w_i | s) $$
They have ...
- 151
1
vote
0
answers
35
views
For the generalised delta rule in back-propogation, do you subtract the target from the obtained output, or vice versa?
When I look up the generalised delta rule equation for back-propogation, I am seeing two conflicting equations.
For example, here (slide 20), given $o$ (the output, defined in slide 18), $z$ (the ...
- 305
0
votes
0
answers
54
views
Neural networks with a continuous function as the input
I am rethinking image recognition algorithms, and want to use an idea I saw in a 3Blue1Brown video:
The basic idea is to map pixels on an image to a specific frequency—every point is distinct—and if ...
- 101
0
votes
0
answers
159
views
Is my calculation of the partial derivative of the cost function with respect to a single weight in the first layer correct?
I'm trying to understand the chain rule of backpropagation.
This is what I understood. Is it correct?
$$ \frac{\partial E }{ \partial w} = \sum_{i} \frac{\partial E }{ \partial a_i^{(l)} } (\sum_{j} \...
0
votes
2
answers
243
views
How the vector-space isomorphism between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$ guarantees reshaping matrices to vectors?
Consider the following paragraph from section 5.4 Gradients fo Matrices of the chapter Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.
Since ...
- 3,990
-1
votes
1
answer
264
views
How can the gradient of the weight be calculated in the viewpoint of matrix calculus?
Let $\sigma(x)$ be sigmoid function. Consider the case where $\text{out}=\sigma(\vec{x} \times W + \vec{b})$, and we want to compute $\frac{\partial{\text{out}}}{\partial{w}
}.$
Set the dimension as ...
- 99