Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gradient-descent]

For questions surrounding gradient descent, a method for finding the optimum state of a parameterized function based on another function often called the loss or error function. It iteratively descends the loss surface to the minimum loss by adjusting parameters based on the product of the partial derivatives comprising the gradient and a learning rate.

2
votes
1answer
35 views

What is the gradient of the objective function in the Soft Actor-Critic paper?

In the paper "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor", they define the loss function for the policy network as $$ J_\pi(\phi)=\mathbb E_{...
1
vote
1answer
41 views

How to obtain a formula for loss, when given an iterative update rule in gradient descent?

From the reinforcement learning book section 13.3: Using pytorch, I need to calculate a loss, and then the gradient is calculated internally. How to obtain the loss from equations which are stated ...
0
votes
0answers
13 views

How to shape the weights or nodes during gradient training of neural network? Training with constraints?

Gradient training changes indiscriminately all the weights and nodes of the neural network. But one can imagine the situations when the training should be shaped, e.g.: One can put constraints on ...
2
votes
1answer
28 views

Will LMS always be convex function? If yes, then why do we change it for neural networks?

In LMS(least mean square) since, we use a quadratic error function, and quadratic functions are generally parabola in (some convex like shape). I wonder whether that is the reason why we use least ...
1
vote
1answer
23 views

Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs?

I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure: ...
6
votes
3answers
131 views

How can a neural network learn when the derivative of the activation function is 0?

Imagine that I have an artificial neural network with a single hidden layer and that I am using ReLU as my activating function. If by change I initialize my bias and my weights in such a form that: $$ ...
0
votes
1answer
40 views

Can gradient descent training be used for nonsmooth loss functions?

I have non-smooth loss function - e.g. loss(x)=min(x, 0.5). Can gradient descent be used for training neural networks with such functions. Can gradient descent be used for fairly general, ...
0
votes
1answer
22 views

Cost function increasing with SGD

In Deep Learning by Goodfellow et al., I came across the following line on the chapter on Stochastic Gradient Descent (pg. 287): The main question is how to set $\epsilon_0$. If it is too large, ...
1
vote
0answers
18 views

neuralnetworksanddeeplearning.com chapter 5 problems

For http://neuralnetworksanddeeplearning.com/chap5.html , could anyone suggest: 1) how to approach the derivation of expression (123) ? 2) what constitutes value ~ 0.45 ? 3) why the need of taylor ...
0
votes
1answer
69 views

Feed forward neural network using numpy for IRIS dataset

I tried to build a neural network for working on IRIS dataset using only numpy after reading an article (link: https://iamtrask.github.io/2015/07/12/basic-python-network/). I tried to search the ...
1
vote
2answers
83 views

SARSA won't work for linear function approximator for MountainCar-v0 in OpenAI environment. What are the possible causes?

I am learning Reinforcement Learning from the lectures from David Silver. I finished lecture 6 and went on to try SARSA with linear function approximator for MountainCar-v0 environment from OpenAI. A ...
1
vote
2answers
53 views

Gradient Descent Feature Scaling

When feature scaling for a Multivariate Linear Regression housing predictor (housing area and # bedrooms), the exercise suggested "scale both types of inputs by their standard deviations and set their ...
0
votes
2answers
181 views

Can the value of a cost function be negative?

I'm new to machine learning and I was watching a video about gradient descent.It said that we want our cost function(Mean squared error) to have the minimum value but that minimum value shown in the ...
1
vote
0answers
24 views

Neural network backpropagation gradient descent better than conjugate gradient descent?

My understanding is that the conjugate gradient method is faster than gradient descent because it does less zig zags while descending. How come the state of the art papers I see all use gradient ...
3
votes
1answer
69 views

Deep Q-Learning: why don't we use mini-batches during experience reply?

In examples and tutorial about DQN, I've often noticed that during the experience replay (training) phase people tend to use stochastic gradient descent / online learning. (e.g. link1, link2) ...
1
vote
3answers
146 views

Batch mode vs mini-batch mode vs stochastic mode

Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size can be one of three options: batch mode: where the ...
1
vote
1answer
39 views

Weight Normalization paper

I am trying to dissect paper about weight normalization: https://papers.nips.cc/paper/6114-weight-normalization-a-simple-reparameterization-to-accelerate-training-of-deep-neural-networks.pdf ...
4
votes
1answer
84 views

Do good approximations produce good gradients?

Let’s say I have a neural net doing classification and I’m doing stochastic gradient descent to train it. If I know that my current approximation is a decent approximation, can I conclude that my ...
4
votes
1answer
362 views

Can non-differentiable layer be used in a neural network, if it's not learned?

For example, AFAIK pooling layer in CNN is not differentiable, but it can be used because it's not learning. Is it always true?
5
votes
1answer
288 views

TRPO/PPO importance sampling term in loss function

In the Trust-Region Policy Optimisation (TRPO) algorithm (and subsequently in PPO also), I do not understand the motivation behind replacing the log probability term from standard policy gradients ...
1
vote
0answers
23 views

Has anyone investigated iteration awareness beyond RNN and LSTM?

This question considers the convergence of an artificial networks (MLPs, RNNs, LSTM nets, CNNs) over time or over the course of epochs made up of iterations through training examples. In this ...
1
vote
2answers
85 views

Mini-batch training and the gradient

I am reading a book that states, "As the mini-batch size increases, the gradient computed is closer to the 'true' gradient". So basically what they are saying is mini-batch training only focuses on ...
0
votes
2answers
47 views

Behaviour of cost

In doing a project using neural networks with an input layer, 4 hidden layers and an output layer ,I used mini batch gradient descent. I noticed that the randomly initialised weights seemed to do a ...
5
votes
2answers
105 views

Working with large datasets

If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini batch gradient descent on this entire dataset is too high, can I ...
2
votes
1answer
68 views

Gradient of boltzmann policy over reward function

I'm struggling with an inverse reinforcement learning problem which seems to appear quite often around the literature, yet I can't find any resources explaining it. The problem is that of calculating ...
1
vote
1answer
42 views

Gradient descent update

In the following, I put the link for the general algorithm of maximum entropy inverse reinforcement learning. http://178.79.149.207/assets/maxent/maxent_slide.jpg This uses a gradient descent ...
2
votes
1answer
170 views

Why Feature Scaling for skewed contour?

Why is it that the skewed contour (unscaled features) will result in slow performance of gradient descent? In other words, how (or why) will the gradients end up taking a long time before finding the ...
1
vote
1answer
40 views

What can be deduced about the “algorithm” of backpropagation/gradient descent?

On this video Link to video a neurologist starts by saying that we do not know how neurons calculate gradients for backpropagation. At minute 30:39 hes showing faster convergence for "our algorithm"...
1
vote
1answer
384 views

How do I implement softmax forward propagation and backpropagation to replace sigmoid in a neural network?

I'm currently using 3Blue1Brown's tutorial series on neural networks and lack extensive calculus knowledge/experience. I'm using the following equations to calculate the gradients for weights and ...
0
votes
1answer
37 views

How to change a weight/bias with gradient

After watching 3Blue1Brown's tutorial series, and an array of others, I'm attempting to make my own neural network from scratch. So far, I'm able to calculate the gradient for each of the weights and ...
1
vote
1answer
323 views

Why use semi-gradient instead of full gradient in RL problems, when using function approximation?

Semi gradient methods work well in Reinforcement Learning, but what is the reason of not using the true gradient if it can be computed? I tried it on the cart pole problem with a deep Q-Network and it ...
3
votes
2answers
559 views

How to perform gradient checking in a neural network with batch normalization?

I have implemented a neural network (NN) using python and numpy only for learning purposes. I have already coded learning rate, momentum, and L1/L2 regularization and checked the implementation with ...
2
votes
0answers
362 views

How to calculate gradient of filter in convolution network

I have similar architecture like in image:CNN. I don't understand how to calculate gradient of filter F. I found these equations(source): Gradient and delta, where first equation calculate gradient ...
4
votes
2answers
318 views

CNN backpropagation with stride>1

I read that to compute the derivative of the error with respect to the input of a convolution layer is the same to make of a convolution between deltas of the next layer and the weight matrix rotated ...
2
votes
2answers
133 views

How is direction of weight change determined by Gradient Descent algorithm

The result of gradient descent algorithm is a vector. So how does this algorithm decide the direction for weight change? We Give hyperparameters for step size. But how is the vector direction for ...
6
votes
2answers
3k views

How is gradient calculated for middle layer weights?

I am trying to understand backpropagation. I used a simple neural network with one input x, one hidden layer h and one output layer y, with weight w1 connecting x to h, and w2 connecting h to y. x--...
1
vote
0answers
36 views

How to calculate Adaptive gradient?

In the FaceNet paper there mentions an gradient algorithm called 'AdaGrad'(Adaptive Gradient) referenced to this paper which has apparently been used to calculate the gradient of the Triplet Loss ...
5
votes
1answer
84 views

What are some concrete steps to deal with the vanishing gradient problem?

So I am training an ANN for classification between 3 classes. The ANN has an input layer, one hidden layer and a 3 node output layer. The problem I am facing is that the output being produced by the ...
0
votes
2answers
108 views

what is the proof behind the gradient of a curve being equal/proportional to the distance between the two co-ordinates in the x-axis [closed]

In the delta rule the equation to adjust the weight with respect to error is :- where is the Learning Rate and E is the ...
6
votes
1answer
338 views

Gradient free training methods for deep learning

As I know, the current state of the art methods for training deep learning networks are variants of gradient descent / stochastic gradient descent. What are the best known gradient-free training ...
7
votes
2answers
385 views

Convexity of MSE in Neural Networks?

I am getting confused reading online about Gradient Descent, Convex and Non Convex Loss functions. Multiple resources I referred to mention that MSE is great because it's convex. But I don't get how, ...
4
votes
1answer
132 views

Can a neural network learn to avoid wrong decisions using backpropagation?

I studied the articles on Neural Networks and Deep Learning from Michael Nielsen and developed a simple neural network based on his examples. I understand how backpropagation works and I already ...
2
votes
0answers
45 views

Are gradients of weights in RNNs dependent on the gradient of every neuron in that layer?

I am writing my own recurrent neural network in Java to understand the inner workings better. While working through the math, I found that in timesteps later than 2 the gradient of weight w of neuron ...
-1
votes
1answer
309 views

How would 1D gradient descent look like?

We have always known that gradient descent is a function of two or more variables. But how can we geometrically represent gradient descent if it is a function of only one variable?
-1
votes
1answer
255 views

Does Musk knows what Gradient descent is

I read a tweet from Elon Musk where describes Gradient descent as an evil action that AI are good at, despite the fact that it is just one of the old, inflexible and not-so-efficient error correction ...
4
votes
1answer
1k views

How to avoid falling into the “local minima” trap? [closed]

How do I avoid my gradient descent algorithm into falling into the "local minima" trap while backpropogating on my neural network? Are there any methods which help me avoid it?