# Questions tagged [calculus]

For questions related to calculus (developed, among others, by Newton and Leibniz), in the context of AI (and, in particular, machine learning).

17 questions
Filter by
Sorted by
Tagged with
17 views

### RNN - Backpropagation through time - Gradient Calculation

I think I got it right after reading multiple resources but im still not 100%. Seems like everyone is calculating it different. Or they just shortcut explaining the calculation. (or my math skills ...
21 views

### How did authors ensure that critical points do exist in GAN?

Using an MLP as a generator introduces multiple critical points in parameter space. You can read this excerpt from the research paper titled Generative Adversarial Nets by Ian J. Goodfellow et al. In ...
33 views

### What are the Calculus books recommended for begineer to advanced researchers in artificial intelligence?

Calculus is a branch of mathematics that deals with the rate of change of outputs of a function w.r.t the inputs. It contains several concepts including limits, first-order derivatives, higher-order ...
60 views

### Why the partial derivative is $0$ when $F_{ij}^l < 0$?. Math behind style transfer

I am currently in the process of reading and understanding the process of style transfer. I came across this equation in the research paper which went like - For context, here is the paragraph - ...
1k views

### Are calculus and differential geometry required for building neural networks?

I've been studying geometry and linear algebra for months with the goal to build neural networks. But now I'm reading that perceptrons require fitting curves, and curves are not expressed as linear ...
50 views

I'm creating a neural network with 3 layers and no bias. On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was: $$\Delta W_{j,k} = (... 1answer 31 views ### How can the gradient of the weight be calculated in the viewpoint of matrix calculus? Let \sigma(x) be sigmoid function. Consider the case where \text{out}=\sigma(\vec{x} \times W + \vec{b}), and we want to compute \frac{\partial{\text{out}}}{\partial{w} }. Set the dimension as ... 0answers 58 views ### BlackOut - ICLR 2016: need help understanding the cost function derivative In the ICLR 2016 paper BlackOut: Speeding up Recurrent Neural Network Language Models with very Large Vocabularies, on page 3, for eq. 4:$$ J_{ml}^s(\theta) = log \ p_{\theta}(w_i | s) $$They have ... 0answers 23 views ### During Backpropagation in LSTM, why is the previous output h_{t-1} considered constant w.r.t any W while computing derivative? I've just started learning LSTM, and some points in the process of calculating the gradients are getting me confused. Say, for example, we want to compute \frac{\partial}{\partial W_i}L, where L ... 0answers 17 views ### For the generalised delta rule in back-propogation, do you subtract the target from the obtained output, or vice versa? When I look up the generalised delta rule equation for back-propogation, I am seeing two conflicting equations. For example, here (slide 20), given o (the output, defined in slide 18), z (the ... 1answer 46 views ### What is the derivative of a specific output with respect to a specific weight? If I have a neural network, and say the 6th output node of the neural network is:$$x_6 = w_{16}y_1 + w_{26}y_2 + w_{36}y_3$$What does that make the derivative of:$$\frac{\partial x_6}{\partial w_{...
I am looking at this formula which breaks down the gradient of $P(\tau |\theta)$ the first part is clear as is the derivative of $\log(x)$, but I do not see how the first formula is rearranged into ...