# Tag Info

Accepted

### Are calculus and differential geometry required for building neural networks?

Neural networks are essentially just repeated matrix multiplications and applications of an activation function, so you really don't need a great deal of linear algebra to construct a simple neural ...
• 970
Accepted

• 278

### Why the partial derivative is $0$ when $F_{ij}^l < 0$?. Math behind style transfer

$F_l$ is the activation of the filter. They state in the paper that they base their method on VGG-Network, which uses ReLU as its activation function. In fact, VGG uses it in all of its hidden layers. ...
• 406

### What are the Calculus books recommended for beginner to advanced researchers in artificial intelligence?

Answer: Calculus James Stewart is the best for a beginner. I started to learn Calculus studying engineering with James Stewart Calculus ( maybe the best for beginners and is really didactic ), ...
1 vote
Accepted

### Which is more popular/common way of representing a gradient in AI community: as a row or column vector?

The issue doesn't come up terribly often. If you are only dealing with vectors, everything is either a row or column vector. It makes no difference which it is. A more relevant issue is whether one ...
• 951
1 vote

### How many directions of gradients exist for a function in higher dimensional space?

Let's look at the definition of gradient: In vector calculus, the gradient of a scalar-valued differentiable function $f$ of several variables is the vector field (or vector-valued function) \$\nabla ...
• 4,273
1 vote

### Are calculus and differential geometry required for building neural networks?

To give some practical advice, it is important to understand parts of calculus. This is mainly because Backpropagation is a leaky abstraction in modern libraries. In a nutshell, there is a lot which ...
• 348
1 vote
Accepted

### How is the log-derivative trick of a trajectory derived?

The identity $$\nabla_{\theta} P(\tau \mid \theta) = P(\tau \mid \theta) \nabla_{\theta} \log P(\tau \mid \theta)\tag{1}\label{1},$$ which can also be written as \begin{align} \nabla_{\theta} \log ...
• 35k
1 vote

### Which linear algebra book should I read to understand vectorized operations?

Linear Algebra Done Right by Axler seems to be the best book on linear algebra, with a brisk and modern approach.
• 1,971
1 vote

### Why is the change in cost wrt bias in neural network equal to error in the neuron?

This is just an application of the chain rule. The same chapter has "Proof of the four fundamental equations" section, which proves BP1-2, while PB3-4 are left as exercise to the reader. I agree that ...
• 1,897

Only top scored, non community-wiki answers of a minimum length are eligible