Questions tagged [gradient]

For questions related to the gradient, a way of packing together all the partial derivative information of a function

Filter by
Sorted by
Tagged with
0 votes
2 answers
52 views

In policy gradient methods why do we compute the gradient of the objective function through a one-trajectory estimate?

Taking as an example the Advantage Actor Critic, the objective function is: \begin{equation} \nabla_{\boldsymbol{\theta}} J(\boldsymbol{\theta})=\mathbb{E}_{\tau \sim \pi_{\boldsymbol{\theta}}}\left[\...
user avatar
1 vote
1 answer
45 views

In multilayer perceptron neural networks, are the names "delta", "gradient" and "error" all the same thing? or not?

What is the difference between "delta", "gradient" and "error", are these names the same thing? I'm confused because someone once told me that both the names "delta&...
will The J's user avatar
0 votes
1 answer
35 views

Gradient: any resource on how to understand everything about it?

I have read some resources about AI, and they all speak about the gradient. Is there any book focused on this? maybe with tons of images / diagrams? Cheers
zerunio's user avatar
1 vote
1 answer
51 views

Why the gradients produced by the soft targets scale as 1/T^2 in knowledge distillation?

In the paper "Distilling the knowledge in a neural network", it mentioned "the magnitudes of the gradients produced by the soft targets scale as 1/(T^2) ", but it has no ...
Xiong's user avatar
  • 11
1 vote
2 answers
55 views

How to apply backpropagation when one layer of the network is a call-only function (no gradient)?

I worked with Feed Forward Neural Network and VAE and understood backpropagation algorithm. Now I build a VAE network, one layer of it is a very complex vector-to-vector function $f(x)$ (a general '...
whitegreen's user avatar
0 votes
1 answer
67 views

Is there a recommended resource that can provide a detailed overview of the gradient norm?

When it comes to the concept of "Gradient Norm," it can be challenging to find a widely recognized and clearly defined resource that offers a comprehensive explanation. While many search ...
StudentV's user avatar
0 votes
0 answers
102 views

Computational overhead of "SCALING FORWARD GRADIENT WITH LOCAL LOSSES"

The paper "SCALING FORWARD GRADIENT WITH LOCAL LOSSES" discusses a new way of training deep neural networks called forward gradient learning. This method is different from the traditional ...
caesar's user avatar
  • 101
3 votes
0 answers
20 views

Why does training converges when the norm of gradient increases?

This is from deep learning book by Ian Goodfellow and Yoshua Bengio and Aaron Courville. When training converges well, I thought the gradient should be at local minima. But the book says it often does ...
tesio's user avatar
  • 195
1 vote
0 answers
123 views

How to estimate the gradient of an argmin loss

Suppose we have a neural network $f_\theta(x)$, where $x$ is the input and $\theta$ is the network's parameters. For each $\theta$, we can minimize $f_\theta(x)$ w.r.t. $x$ and obtain the minimum ...
Mingzhou Liu's user avatar
2 votes
0 answers
30 views

How to prepare audio data for deep learning?

Audio data is typically an array with the waveform represented by values from -1 to 1. There are two issues with that: if all values are inverted, e.g. -1 becomes 1 and 1 becomes -1, the audio doesn'...
nikishev.'s user avatar
2 votes
0 answers
325 views

GAN : Why does a perfect discriminator mean no gradient for the generator?

In the training of a Generative Adversarial Networks (GAN) system, a perfect discriminator (D) is one which outputs 1 ("true image") for all images of the training dataset and 0 ("false ...
Soltius's user avatar
  • 221
0 votes
1 answer
1k views

Why to use gradient accumulation?

I know that gradient accumulation is (1) a way to reduce memory usage while still enabling the machine to fit a large dataset (2) reducing the noise of the gradient compared to SGD, and thus smoothing ...
Charles's user avatar
  • 101
3 votes
1 answer
114 views

Why is automatic differentiation still used, if today's computers can calculate symbolic derivatives quite fast?

Today's computers can calculate symbolic derivatives quite fast, why is automatic differentiation still used? For example, Mathematica can handle algebraic operations with arrays. Doesn't automatic ...
asd's user avatar
  • 33
-1 votes
1 answer
137 views

I’m making a simple neural network from scratch and it won’t learn anything. Please help [closed]

I am coding a classifier neural network from scratch. It is not really learning and I believe that somewhere there is a gradient explosion/vanishing issue. Could be some other stuff as well that I ...
Maks's user avatar
  • 31
0 votes
0 answers
52 views

Basic question about gradient for nominal regression

Say that we want to binary-classify images using a sigmoid function with the entropy-loss function. This algorithm is quite slow. The sigmoid function is: I find that this could be traced to the $L(y,...
Mah Neh's user avatar
  • 79
1 vote
1 answer
527 views

How does MAML inner loop optimization works?

I started to learn meta-learning, reading the MAML paper https://arxiv.org/pdf/1703.03400.pdf In the inner loop, I am calculating adapted parameters for each task, I will be doing multiple steps of ...
Grumpy C's user avatar
1 vote
0 answers
36 views

What do "large variables" and "small weights" mean in these sentences?

I'm trying to understand these two points from an article: Models with large variables i.e weight matrices. As a consequence such models have correspondingly large gradients and optimizer states. The ...
NRain's user avatar
  • 111
0 votes
1 answer
343 views

How to explain near zero gradients on first epochs?

As I understand the gradient should reflect how near the weights are to the optimal values. In this way i will expect that on the first epochs the gradients far from zero or at least not mostly zero ...
Guillermo Alvarez's user avatar
1 vote
0 answers
345 views

Why would one prefer the gradient of the sum rather than the sum of the gradients?

When gradients are aggregated over mini batches, I sometimes see formulations like this, e.g., in the "Deep Learning" book by Goodfellow et al. $$\mathbf{g} = \frac{1}{m} \nabla_{\mathbf{w}} ...
Eddie C's user avatar
  • 11
3 votes
2 answers
3k views

What does it mean by "gradient flow" in the context of neural networks?

Several research papers and textbooks (e.g. this) contain the phrase "gradient flow" in the context of neural networks. I am confused about whether it has any rigorous and formal way of ...
hanugm's user avatar
  • 3,612
2 votes
2 answers
837 views

What specifically is the gradient of the log of the probability in policy gradient methods?

I am getting tripped up slightly by how specifically the gradient is calculated in policy gradient methods (just the intuitive understanding of it). This Math Stack Exchange post is close, but I'm ...
user9317212's user avatar
1 vote
0 answers
355 views

What exactly do gradient-based saliency map tell us?

As far as I understand, gradients are supposed to tell us 1) the magnitude and 2) direction, to update a parameter such as to minimize the loss function. Regarding saliency maps, which use gradients ...
user452306's user avatar
1 vote
1 answer
69 views

Rank of gradient-of-loss with respect to layer weights in an MLP

The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3: The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
Andrew's user avatar
  • 63
0 votes
1 answer
41 views

Which is more popular/common way of representing a gradient in AI community: as a row or column vector?

Consider the following remark about writing gradients from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al. Remark (...
hanugm's user avatar
  • 3,612
0 votes
1 answer
131 views

What is the rigorous and formal definition for the direction pointed by a gradient?

Consider the following definition of derivative from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al. Definition 5.2 (...
hanugm's user avatar
  • 3,612
1 vote
1 answer
237 views

Mathematically speaking, Is it only the product operation used in the chain rule causing the vanishing or exploding gradient?

I am asking this question from the mathematical perspective of the vanishing and exploding gradient problems that we face generally during training deep neural networks. The chain rule of ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
165 views

Isssue in understanding the derivation regarding mean squared error

The following derivation is taken from Chapter 5: Machine Learning Basics from the book titled Deep Learning (by Aaron Courville et al.) I am facing difficulty in understanding the zero derivative ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
292 views

How many directions of gradients exist for a function in higher dimensional space?

Gradients are used in optimization algorithms. Based on the values of gradients, we generally update the weights of a neural network. It is known that gradients have a direction and the direction ...
hanugm's user avatar
  • 3,612
1 vote
1 answer
81 views

What all does the gradient tells us other than the direction to move parameters?

Gradients are used in optimization algorithms. I know that a gradient gives us information about the direction in which one needs to update the weights of a neural network. We need to travel in the ...
hanugm's user avatar
  • 3,612
1 vote
1 answer
41 views

What is the high-level algorithm followed by contemporary packages for the calculation of gradient?

Most of the neural network models in contemporary deep learning packages are trained based on gradients. Let $f: \mathbb{R}^m \rightarrow \mathbb{R}^n$ be a function for which we want to find a ...
hanugm's user avatar
  • 3,612
1 vote
1 answer
66 views

Is there any significance for higher order gradients in artificial intelligence?

Although I don't know in detail, I am aware of the following facts regarding the use of gradients in some domains of artificial intelligence, especially in minimizing the training of neural networks. ...
hanugm's user avatar
  • 3,612
2 votes
1 answer
236 views

What does it mean by strong or sufficient gradient for training in this context?

It has been mentioned in the research paper titled Generative Adversarial Nets that generator need to maximize the function $\log D(G(z))$ instead of minimizing $\log(1 −D(G(z)))$ since the former ...
hanugm's user avatar
  • 3,612
1 vote
1 answer
128 views

What is meant by "well-behaved gradient" in this context?

Consider the following statement (from the paper Generative Adversarial Nets) about the success of discriminative models So far, the most striking successes in deep learning have involved ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
723 views

How to calculate the gradient penalty proposed in "Improved Training of Wasserstein GANs"?

The research paper titled Improved Training of Wasserstein GANs proposed a gradient penalty in order to avoid undesired behavior due to weight clipping of the discriminator. We now propose an ...
hanugm's user avatar
  • 3,612
1 vote
1 answer
127 views

What does it mean by "zeros the networks parameters gradients" in the context of training a neural network?

Consider the following PyTorch code ...
hanugm's user avatar
  • 3,612
1 vote
0 answers
17 views

Terminology for the weight of likelihood ratio/score function?

If we estimate the gradient of $f(x)$ using the likelihood ratio/score function, i.e. $$\nabla f = f^*\dfrac{\partial \log p(x)}{\partial \theta}$$ is there any agreed upon terminology to call "$...
Taw's user avatar
  • 1,201
0 votes
1 answer
161 views

Why don't integrated gradients explain samples correctly?

I have a linear tabular dataset made of floats. The dataset follows a simple rule like: ...
user46784's user avatar
4 votes
1 answer
2k views

Why is it a problem if the outputs of an activation function are not zero-centered?

In this lecture, the professor says that one problem with the sigmoid function is that its outputs aren't zero-centered. Are the explanation provided by the professor regarding why this is bad is that ...
Daviiid's user avatar
  • 573
5 votes
2 answers
1k views

Why is tf.abs non-differentiable in Tensorflow?

I understand why tf.abs is non-differentiable in principle (discontinuity at 0) but the same applies to tf.nn.relu yet, in case of this function gradient is simply set to 0 at 0. Why the same logic is ...
zedsdead's user avatar
2 votes
1 answer
104 views

What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?

I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-...
Blake Camp's user avatar
11 votes
1 answer
11k views

What is the relationship between gradient accumulation and batch size?

I am currently training some models using gradient accumulation since the model batches do not fit in GPU memory. Since I am using gradient accumulation, I had to tweak the training configuration a ...
JVGD's user avatar
  • 1,098
2 votes
2 answers
1k views

How can we compute the gradient of max pooling with overlapping regions?

While studying backpropagation in CNNs, I can't understand how can we compute the gradient of max pooling with overlapping regions. That's also a question from this quiz and can be also found on this ...
estamos's user avatar
  • 157
5 votes
2 answers
329 views

Is the gradient at a layer independent of the activations of the previous layers?

Is the gradient at a layer (of a feed-forward neural network) independent of the activations of the previous layers? I read this in a paper titled Mean Field Residual Networks: On the Edge of Chaos (...
Snehal Reddy's user avatar
6 votes
1 answer
2k views

How is the gradient of the loss function in DQN derived?

In the original DQN paper, page 1, the loss function of the DQN is $$ L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim U(D)} [(r+\gamma \max_{a'} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}))^2] $$ ...
Dimitris Monroe's user avatar
1 vote
1 answer
420 views

How can the sum of squared errors have negative gradient if it's defined as the squared of the error?

The formula for the sum of squared errors (SSE) is: $$ \frac{1}{2} \sum_{i=1}^n (t^i - o^i)^2 $$ I have a few related questions. If $t^i - o^i$ is negative, doesn't the power of 2 eliminate any ...
Ben's user avatar
  • 425
5 votes
2 answers
618 views

Why is the derivative of this objective function 0 if the policy is deterministic?

In the Berkeley RL class CS294-112 Fa18 9/5/18, they mention the following gradient would be 0 if the policy is deterministic. $$ \nabla_{\theta} J(\theta)=E_{\tau \sim \pi_{\theta}(\tau)}\left[\left(\...
jonperl's user avatar
  • 153