# Questions tagged [calculus]

For questions related to calculus (developed, among others, by Newton and Leibniz), in the context of AI (and, in particular, machine learning).

27 questions
Filter by
Sorted by
Tagged with
51 views

### Neural networks with a continuous function as the input

I am rethinking image recognition algorithms, and want to use an idea I saw in a 3Blue1Brown video: The basic idea is to map pixels on an image to a specific frequency—every point is distinct—and if ...
• 101
10 views

### Equivalent of bivariate Fourier series with polynomial denominator

My goal is to have an equivalent, as $t\to \infty$, of $$\sum_{m\in \mathbb{Z}^2,m\neq 0}\frac{1}{\|m\|^\alpha}\exp(im\cdot u/t)\exp(-(m/t)^2)$$ for $u\in \mathbb{R}^2$. Does anyone recognize ...
• 101
111 views

### What is the partial derivative $\frac{\partial y}{\partial x_1}$ in this neural network?

The answer is supposed to be -6, but I don't know how to get that. Also, in a NN, is that 2nd hidden layer possible, where the neurons are not dependent on all the neurons of the previous layer?
1 vote
133 views

### How do policy gradients work?

If I understand it correctly from the following equation $$U(\theta)=\mathbb{E}_{\tau \sim P(\tau;\theta)}\left [ \sum_{t=0}^{H-1}R(s_t,u_t);\pi_{\theta} \right ]=\sum_{\tau}P(\tau;\theta)R(\tau)$$ ...
• 175
180 views

### Best calculus books for Deep Learning

Recommend some calculus books for Deep Learning and neural networks. I know what is integration, differentiation, derivates, limits on a based level. I would like to understand on deep level the ...
• 31
154 views

243 views

### How can the gradient of the weight be calculated in the viewpoint of matrix calculus?

Let $\sigma(x)$ be sigmoid function. Consider the case where $\text{out}=\sigma(\vec{x} \times W + \vec{b})$, and we want to compute $\frac{\partial{\text{out}}}{\partial{w} }.$ Set the dimension as ...
• 99
1 vote
82 views

### BlackOut - ICLR 2016: need help understanding the cost function derivative

In the ICLR 2016 paper BlackOut: Speeding up Recurrent Neural Network Language Models with very Large Vocabularies, on page 3, for eq. 4: $$J_{ml}^s(\theta) = log \ p_{\theta}(w_i | s)$$ They have ...
• 151
1 vote
35 views

### For the generalised delta rule in back-propogation, do you subtract the target from the obtained output, or vice versa?

When I look up the generalised delta rule equation for back-propogation, I am seeing two conflicting equations. For example, here (slide 20), given $o$ (the output, defined in slide 18), $z$ (the ...
• 297
534 views

• 121
291 views

### Is Gradient Descent algorithm a part of Calculus of Variations?

As in https://en.wikipedia.org/wiki/Calculus_of_variations The calculus of variations is a field of mathematical analysis that uses variations, which are small changes in functions and ...
• 1,283
219 views

### Which linear algebra book should I read to understand vectorized operations?

I am reading Goodfellow's book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this ...
• 151