Questions tagged [gradient]
For questions related to the gradient, a way of packing together all the partial derivative information of a function
29
questions
2
votes
2answers
67 views
What specifically is the gradient of the log of the probability in policy gradient methods?
I am getting tripped up slightly by how specifically the gradient is calculated in policy gradient methods (just the intuitive understanding of it).
This Math Stack Exchange post is close, but I'm ...
1
vote
0answers
48 views
What exactly do gradient-based saliency map tell us?
As far as I understand, gradients are supposed to tell us 1) the magnitude and 2) direction, to update a parameter such as to minimize the loss function.
Regarding saliency maps, which use gradients ...
0
votes
0answers
20 views
Rank of gradient-of-loss with respect to layer weights in an MLP
The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3:
The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
0
votes
1answer
31 views
Which is more popular/common way of representing a gradient in AI community: as a row or column vector?
Consider the following remark about writing gradients from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.
Remark (...
0
votes
1answer
61 views
What is the rigorous and formal definition for the direction pointed by a gradient?
Consider the following definition of derivative from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.
Definition 5.2 (...
1
vote
1answer
41 views
Mathematically speaking, Is it only the product operation used in the chain rule causing the vanishing or exploding gradient?
I am asking this question from the mathematical perspective of the vanishing and exploding gradient problems that we face generally during training deep neural networks.
The chain rule of ...
0
votes
1answer
27 views
Isssue in understanding the derivation regarding mean squared error
The following derivation is taken from Chapter 5: Machine Learning Basics from the book titled Deep Learning (by Aaron Courville et al.)
I am facing difficulty in understanding the zero derivative ...
0
votes
1answer
66 views
How many directions of gradients exist for a function in higher dimensional space?
Gradients are used in optimization algorithms. Based on the values of gradients, we generally update the weights of a neural network.
It is known that gradients have a direction and the direction ...
0
votes
0answers
33 views
What all does the gradient tells us?
Gradients are used in optimization algorithms.
I know that a gradient gives us information about the direction in which one needs to update the weights of a neural network. We need to travel in the ...
1
vote
1answer
33 views
What is the high-level algorithm followed by contemporary packages for the calculation of gradient?
Most of the neural network models in contemporary deep learning packages are trained based on gradients.
Let $f: \mathbb{R}^m \rightarrow \mathbb{R}^n$ be a function for which we want to find a ...
0
votes
0answers
30 views
Is there any significance for higher order gradients in artificial intelligence?
Although I don't know in detail, I am aware of the following facts regarding the use of gradients in some domains of artificial intelligence, especially in optimization.
First order gradient: It ...
2
votes
1answer
53 views
What does it mean by strong or sufficient gradient for training in this context?
It has been mentioned in the research paper titled Generative Adversarial Nets that generator need to maximize the function $\log D(G(z))$ instead of minimizing $\log(1 −D(G(z)))$ since the former ...
0
votes
0answers
23 views
How to handle critical points during generator training?
Using an MLP as a generator introduces multiple critical points in parameter space. You can read this excerpt from the research paper titled Generative Adversarial Nets by Ian J. Goodfellow et al.
In ...
1
vote
1answer
53 views
What is meant by "well-behaved gradient" in this context?
Consider the following statement (from the paper Generative Adversarial Nets) about the success of discriminative models
So far, the most striking successes in deep learning have involved ...
0
votes
1answer
39 views
How to calculate the gradient penalty proposed in "Improved Training of Wasserstein GANs"?
The research paper titled Improved Training of Wasserstein GANs proposed a gradient penalty in order to avoid undesired behavior due to weight clipping of the discriminator.
We now propose an ...
0
votes
0answers
76 views
What are the input and output gradients in PyTorch?
Suppose I want to train a neural network with $m-$length inputs of form $x = [x_1, x_2, x_3, \cdots, x_m]$ and $n-$length outputs of form $y = [y_1, y_2, y_3, \cdots, y_n]$.
Let the number of ...
1
vote
1answer
43 views
What does it mean by "zeros the networks parameters gradients" in the context of training a neural network?
Consider the following PyTorch code
...
0
votes
0answers
20 views
Gradient of Scalar objective cannot be efficiently calculated?
Suppose we generate the vector output $y$ from model $h(x, \theta)$, with input $x$ and parameters $\theta$. Reverse mode differentiation says that we can calculate the gradient
\begin{align*}
\...
1
vote
0answers
15 views
Terminology for the weight of likelihood ratio/score function?
If we estimate the gradient of $f(x)$ using the likelihood ratio/score function, i.e.
$$\nabla f = f^*\dfrac{\partial \log p(x)}{\partial \theta}$$
is there any agreed upon terminology to call "$...
0
votes
1answer
66 views
Why don't integrated gradients explain samples correctly?
I have a linear tabular dataset made of floats. The dataset follows a simple rule like:
...
3
votes
1answer
163 views
Why is it a problem if the outputs of an activation function are not zero-centered?
In this lecture, the professor says that one problem with the sigmoid function is that its outputs aren't zero-centered. Are the explanation provided by the professor regarding why this is bad is that ...
5
votes
2answers
265 views
Why is tf.abs non-differentiable in Tensorflow?
I understand why tf.abs is non-differentiable in principle (discontinuity at 0) but the same applies to tf.nn.relu yet, in case of this function gradient is simply set to 0 at 0. Why the same logic is ...
2
votes
1answer
74 views
What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?
I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
7
votes
1answer
4k views
What is the relationship between gradient accumulation and batch size?
I am currently training some models using gradient accumulation since the model batches do not fit in GPU memory. Since I am using gradient accumulation, I had to tweak the training configuration a ...
2
votes
2answers
841 views
How can we compute the gradient of max pooling with overlapping regions?
While studying backpropagation in CNNs, I can't understand how can we compute the gradient of max pooling with overlapping regions.
That's also a question from this quiz and can be also found on this ...
5
votes
2answers
100 views
Is the gradient at a layer independent of the activations of the previous layers?
Is the gradient at a layer (of a feed-forward neural network) independent of the activations of the previous layers?
I read this in a paper titled Mean Field Residual Networks: On the Edge of Chaos (...
4
votes
1answer
1k views
How is the gradient of the loss function in DQN derived?
In the original DQN paper, page 1, the loss function of the DQN is
$$
L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim U(D)} [(r+\gamma \max_{a'} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}))^2]
$$
...
1
vote
1answer
164 views
How can the sum of squared errors have negative gradient if it's defined as the squared of the error?
The formula for the sum of squared errors (SSE) is:
$$
\frac{1}{2} \sum_{i=1}^n (t^i - o^i)^2
$$
I have a few related questions.
If $t^i - o^i$ is negative, doesn't the power of 2 eliminate any ...
4
votes
2answers
542 views
Why is the derivative of this objective function 0 if the policy is deterministic?
In the Berkeley RL class CS294-112 Fa18 9/5/18, they mention the following gradient would be 0 if the policy is deterministic.
$$
\nabla_{\theta} J(\theta)=E_{\tau \sim \pi_{\theta}(\tau)}\left[\left(\...
