# Questions tagged [calculus]

For questions related to calculus (developed, among others, by Newton and Leibniz), in the context of AI (and, in particular, machine learning).

12 questions
Filter by
Sorted by
Tagged with
88 views

### Which linear algebra book should I read to understand vectorized operations?

I am reading Goodfellow's book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this ...
100 views

### Why is the change in cost wrt bias in neural network equal to error in the neuron?

While reading the book on neural networks by Michael Nielson, I had a problem understanding equation (BP3), which is $$\frac{\partial C}{\partial b_{j}^{l}}=\delta_{j}^{l} \tag{BP3}\label{BP3},$$ ...
1k views

### What is a bad local minimum in machine learning?

What is "bad local minima"? The following papers all mention this expression. Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit limination of All Bad Local ...
115 views

### How is the log-derivative trick of a trajectory derived?

I am looking at this formula which breaks down the gradient of $P(\tau |\theta)$ the first part is clear as is the derivative of $\log(x)$, but I do not see how the first formula is rearranged into ...
30 views

41 views

### What is the correct formula for updating the weights in a 1-single hidden layer neural network?

I'm creating a neural network with 3 layers and no bias. On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was: \Delta W_{j,k} = (...
### During Backpropagation in LSTM, why is the previous output $h_{t-1}$ considered constant w.r.t any $W$ while computing derivative?
I've just started learning LSTM, and some points in the process of calculating the gradients are getting me confused. Say, for example, we want to compute $\frac{\partial}{\partial W_i}L$, where $L$ ...
Let $\sigma(x)$ be sigmoid function. Consider the case where $\text{out}=\sigma(\vec{x} \times W + \vec{b})$, and we want to compute $\frac{\partial{\text{out}}}{\partial{w} }.$ Set the dimension as ...