Skip to main content
13 votes
Accepted

Why is the derivative of this objective function 0 if the policy is deterministic?

Here is the gradient that they are discussing in the video: $$\nabla_{\theta} J(\theta) \approx \frac{1}{N} \sum_{i=1}^N \left( \sum_{t=1}^T \nabla_{\theta} \log \pi_{\theta} (\mathbf{a}_{i, t} \vert \...
Dennis Soemers's user avatar
  • 10.4k
13 votes
Accepted

Are calculus and differential geometry required for building neural networks?

Neural networks are essentially just repeated matrix multiplications and applications of an activation function, so you really don't need a great deal of linear algebra to construct a simple neural ...
htl's user avatar
  • 1,010
3 votes
Accepted

What is the partial derivative $\frac{\partial y}{\partial x_1}$ in this neural network?

From calculus rules: $\frac{\partial y}{\partial x_1}=\frac{\partial y}{\partial r_1}\frac{\partial r_1}{\partial x_1}+\frac{\partial y}{\partial r_2}\frac{\partial r_2}{\partial x_1}=3c+d=-6$. In a ...
cinch's user avatar
  • 2,542
3 votes
Accepted

Which linear algebra book should I read to understand vectorized operations?

If you already have two years of a bachelor's of mathematics, I recommend part I of the book that you're mentioning. That part of the book reviews the main mathematics used in the optimization of ...
3 votes

Why is the derivative of this objective function 0 if the policy is deterministic?

Well, I'd rather comment, but I don't have yet this privilege, so here are some comments. First, having a deterministic policy inside the log would do create trivial terms. Secondly, for me, in ...
16Aghnar's user avatar
  • 601
2 votes
Accepted

Are my computations of the forward and backward pass of a neural network with one input, hidden and output neurons correct?

One important point I missed in first review: error is a summatory, its derivative is also a summatory. About offsets "b": usually they are different in each cell (if not fixed to some value, as 0). ...
pasaba por aqui's user avatar
2 votes
Accepted

What is the derivative of a specific output with respect to a specific weight?

Formally speaking $x_6$ is a function of $w_{16},\ w_{26}$ and $w_{36}$, that is $$x_6 =f(w_{16}, w_{26}, w_{36})=w_{16}y_1 + w_{26}y_2 + w_{36}y_3.$$ The derivative w.r.t. $w_{26}$ is $$\frac{\...
Uskebasi's user avatar
  • 278
2 votes
Accepted

How is the log-derivative trick of a trajectory derived?

The identity $$\nabla_{\theta} P(\tau \mid \theta) = P(\tau \mid \theta) \nabla_{\theta} \log P(\tau \mid \theta)\tag{1}\label{1},$$ which can also be written as \begin{align} \nabla_{\theta} \log ...
nbro's user avatar
  • 41k
2 votes

Why the partial derivative is $0$ when $F_{ij}^l < 0$?. Math behind style transfer

$F_l$ is the activation of the filter. They state in the paper that they base their method on VGG-Network, which uses ReLU as its activation function. In fact, VGG uses it in all of its hidden layers. ...
Avatrin's user avatar
  • 496
2 votes
Accepted

How do policy gradients work?

From the equation you wrote: $U(\theta)$ is the total expected reward of a policy $\pi_\theta$ (with parameters/weights $\theta$) over all possible trajectories $\tau$ sampled under the environment ...
Luca Anzalone's user avatar
2 votes
Accepted

What is a bad local minimum in machine learning?

The adjective bad isn't mathematically descriptive. A better term is sub-optimal, which implies the state of learning might appear optimal based on current information but the optimal solution from ...
Douglas Daseeco's user avatar
2 votes

What are the Calculus books recommended for beginner to advanced researchers in artificial intelligence?

Answer: Calculus James Stewart is the best for a beginner. I started to learn Calculus studying engineering with James Stewart Calculus ( maybe the best for beginners and is really didactic ), ...
rubengavidia0x's user avatar
1 vote

How many directions of gradients exist for a function in higher dimensional space?

Let's look at the definition of gradient: In vector calculus, the gradient of a scalar-valued differentiable function $f$ of several variables is the vector field (or vector-valued function) $\nabla ...
Edoardo Guerriero's user avatar
1 vote

What all does the gradient tells us other than the direction to move parameters?

Momentum was big. It allowed several steps to be evened out so that most of the motion in the weights was in the direction of the optimum. It operates against sequential measurements of the error. ...
EngrStudent's user avatar
1 vote
Accepted

Which is more popular/common way of representing a gradient in AI community: as a row or column vector?

The issue doesn't come up terribly often. If you are only dealing with vectors, everything is either a row or column vector. It makes no difference which it is. A more relevant issue is whether one ...
Taw's user avatar
  • 1,291
1 vote

What is the rigorous and formal definition for the direction pointed by a gradient?

If $u$ is a vector, the direction pointed by the vector is defined as $\dfrac{u}{\lVert {u}\rVert}$ where $\lVert \cdot \rVert$ is the 2 norm (euclidean norm).
Taw's user avatar
  • 1,291
1 vote

Are calculus and differential geometry required for building neural networks?

To give some practical advice, it is important to understand parts of calculus. This is mainly because Backpropagation is a leaky abstraction in modern libraries. In a nutshell, there is a lot which ...
tnfru's user avatar
  • 348
1 vote

What is the correct formula for updating the weights in a 1-single hidden layer neural network?

The correct formula for updating the weights between the hidden layer and the output layer is: $$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$ where $h$ is the activated hidden layer ...
Tiago Cavalcante's user avatar
1 vote

Which linear algebra book should I read to understand vectorized operations?

Linear Algebra Done Right by Axler seems to be the best book on linear algebra, with a brisk and modern approach.
k.c. sayz 'k.c sayz''s user avatar
1 vote

Why is the change in cost wrt bias in neural network equal to error in the neuron?

This is just an application of the chain rule. The same chapter has "Proof of the four fundamental equations" section, which proves BP1-2, while PB3-4 are left as exercise to the reader. I agree that ...
Maxim's user avatar
  • 1,967

Only top scored, non community-wiki answers of a minimum length are eligible