# Questions tagged [calculus]

For questions related to calculus (developed, among others, by Newton and Leibniz), in the context of AI (and, in particular, machine learning).

26 questions
Filter by
Sorted by
Tagged with
7 views

### Equivalent of bivariate Fourier series with polynomial denominator

My goal is to have an equivalent, as $t\to \infty$, of $$\sum_{m\in \mathbb{Z}^2,m\neq 0}\frac{1}{\|m\|^\alpha}\exp(im\cdot u/t)\exp(-(m/t)^2)$$ for $u\in \mathbb{R}^2$. Does anyone recognize ...
101 views

### What is the partial derivative $\frac{\partial y}{\partial x_1}$ in this neural network?

The answer is supposed to be -6, but I don't know how to get that. Also, in a NN, is that 2nd hidden layer possible, where the neurons are not dependent on all the neurons of the previous layer?
1 vote
109 views

### How do policy gradients work?

If I understand it correctly from the following equation $$U(\theta)=\mathbb{E}_{\tau \sim P(\tau;\theta)}\left [ \sum_{t=0}^{H-1}R(s_t,u_t);\pi_{\theta} \right ]=\sum_{\tau}P(\tau;\theta)R(\tau)$$ ...
127 views

### Best calculus books for Deep Learning

Recommend some calculus books for Deep Learning and neural networks. I know what is integration, differentiation, derivates, limits on a based level. I would like to understand on deep level the ...
148 views

227 views

### How can the gradient of the weight be calculated in the viewpoint of matrix calculus?

Let $\sigma(x)$ be sigmoid function. Consider the case where $\text{out}=\sigma(\vec{x} \times W + \vec{b})$, and we want to compute $\frac{\partial{\text{out}}}{\partial{w} }.$ Set the dimension as ...
1 vote
81 views

### BlackOut - ICLR 2016: need help understanding the cost function derivative

In the ICLR 2016 paper BlackOut: Speeding up Recurrent Neural Network Language Models with very Large Vocabularies, on page 3, for eq. 4: $$J_{ml}^s(\theta) = log \ p_{\theta}(w_i | s)$$ They have ...
1 vote
34 views

### For the generalised delta rule in back-propogation, do you subtract the target from the obtained output, or vice versa?

When I look up the generalised delta rule equation for back-propogation, I am seeing two conflicting equations. For example, here (slide 20), given $o$ (the output, defined in slide 18), $z$ (the ...
437 views

270 views

### Is Gradient Descent algorithm a part of Calculus of Variations?

As in https://en.wikipedia.org/wiki/Calculus_of_variations The calculus of variations is a field of mathematical analysis that uses variations, which are small changes in functions and ...
205 views

### Which linear algebra book should I read to understand vectorized operations?

I am reading Goodfellow's book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this ...