Questions tagged [gradient-descent]

For questions surrounding gradient descent, a method for finding the optimum state of a parameterized function based on another function often called the loss or error function. It iteratively descends the loss surface to the minimum loss by adjusting parameters based on the product of the partial derivatives comprising the gradient and a learning rate.

Filter by
Sorted by
Tagged with
3
votes
1answer
56 views

Eligibility vector for softmax policy with policy gradients

There is this nice result for policy gradients that the gradient of some performance measure, $\nabla v_{\pi_{\theta}}(s_0)$ (here, in the episodic case for the starting state $s_0$ and policy $\pi$, ...
2
votes
1answer
28 views

How to reduce variance of the model loss during training?

I know that stochastic gradient descent always gives different results. What are the best practices to reduce this variance today? I tried to predict simple function with two different approaches and ...
1
vote
0answers
10 views

Can Grad CAM feature maps be used for Training?

I am trying to recreate the architecture of the following paper: https://arxiv.org/pdf/1807.03058.pdf Can someone help me in explaining how are the feature maps coming out of the output of GradCam ...
1
vote
0answers
30 views

Understanding the partial derivative with respect to the weight matrix and bias

Say we have the layer $X W + b = Y$. I want to get $\frac{dL}{dW}$ and we assume I have $\frac{dL}{dY}$. So all I need is to find $\frac{dY}{dW}$. I know that it should be $X^T\frac{dL}{dY}$ but don'...
1
vote
1answer
33 views

How to plot Loss Landscape with more than 2 weights in the network

For a single neuron with 2 weights, I can plot the loss landscape and it looks like this (OR data, sigmoid activation, MAE loss): But, when the neuron accepts more inputs, which means more than 2 ...
4
votes
2answers
34 views

Is the gradient at a layer independent of the activations of the previous layers?

Is the gradient at a layer (of a feed-forward neural network) independent of the activations of the previous layers? I read this in a paper titled Mean Field Residual Networks: On the Edge of Chaos (...
0
votes
1answer
26 views

Backpropagation: Chain Rule to the Third Last Layer

I'm trying to solve dLoss/dW1. The network is as in picture below with identity activation at all neurons: Solving dLoss/dW7 is simple as there's only 1 way to output: $Delta = Out-Y$ $Loss = abs(...
2
votes
0answers
19 views

How to calculate multiobjective optimization cost for ordinary problems?

What I did: Created a population of 2D legged robots in a simulated environment. Found the best motor rotation values to make the robots move rightward, using an objective function with Differential ...
3
votes
1answer
36 views

Why is batch gradient descent performing worse than stochastic and minibatch gradient descent?

I have implemented a neural network from scratch (only using numpy) and I am having problems understanding why the results are so different between stochastic/minibatch gradient descent and batch ...
0
votes
1answer
37 views

Network doesn't converge with ReLU or Leaky ReLU, but works well with sigmoid/tanh

I have these training data to separate, the classes are rather randomly scattered: My first attempt was using tf.nn.relu activation function, but output was stuck ...
1
vote
1answer
151 views

Online Learning for Neural Networks

There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform online learning with neural ...
4
votes
1answer
79 views

An intuitive explanation of Adagrad, its purpose and its formula

It (Adagrad) adapts the learning rate to the parameters, performing smaller updates (i.e. low learning rates) for parameters associated with frequently occurring features, and larger updates (i.e. ...
2
votes
0answers
48 views

Does Retina-net's focal loss accomplish its goal?

Taking out the weighting factor we can define focal loss as $$FL(p) = -(1-p)^\gamma log(p) $$ Where $p$ is the target probability. The idea being that single stage object detectors have a huge ...
0
votes
1answer
39 views

Is it possible with stochastic gradient descent for the error to increase?

As simple as that. Is there any scenario where the error might increase, if only by a tiny amount, when using SGD (no momentum)?
4
votes
2answers
188 views

Can neuroevolution be combined with gradient descent?

Is there any precedent for using a neuroevolution algorithm, like NEAT, as a way of getting to an initialization of weights for a network that can then be fine-tuned with gradient descent and back-...
1
vote
1answer
32 views

In NN, as iterations of Gradient descent increases, the accuracy of Test/CV set decreases. how can i resolve this?

As mentioned in the title I'm using 300 Dataset example with 500 feature as an input. As I'm training the dataset, I found something peculiar. Please look at the data shown below. Iteration 5000 |...
1
vote
1answer
58 views

Sensitivity of neural network to inputs

I am trying to build intution regarding neural networks and their working and this is a question, I am interested in: I understand that we normalise inputs. The reason this is done is to capture the ...
2
votes
0answers
53 views

Neural networks when gradient descent is not possible

I am looking for an example in which it is simply impossible to use some sort of gradient descent to train a neural network. Is this available? I have read quite some papers about gradient-free ...
1
vote
1answer
72 views

How does NEAT find the most successful generation without gradients?

I'm new to NEAT, so, please, don't be too harsh. How does NEAT find the most successful generation without gradient descent or gradients?
4
votes
3answers
193 views

Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The MSE can be defined as $(\hat{y} - y)^2$, which should be equal to $(y - \hat{y})^2$, but I think their derivative is different, so I am confused of what derivative will I use for computing my ...
2
votes
0answers
41 views

What is the right formula for weight update rule in Logistic Regression using stochastic gradient descent

Apologies for the lengthy title. My question is about the weight update rule for logistic regression using stochastic gradient descent. I have just started experimenting on Logistic Regression. I ...
6
votes
2answers
269 views

How is local minima possible in gradient descent?

Gradient descent works on the equation of mean squared error, which is an equation of a parabola $y=x^2$. We often say that weight adjustment in a neural network by gradient descent algorithm can hit ...
2
votes
1answer
54 views

How can we reach global optimum?

Gradient descent can get stuck into local optimum. Which techniques are there to reach global optimum?
1
vote
1answer
69 views

When is bias values updated in back propagation?

I am new to deep learning. I have doubts on modifying bias values during back propagation. My doubts are Does the back propagation algorithm modifies the weigh values and bias values in the same pass?...
1
vote
1answer
137 views

Is back propagation applied for each data point or for a batch of data points?

I am new to deep learning and trying to understand the concept of back propagation. I have a doubt on when the back propagation is applied. Assume that I have a training data set of 1000 images for ...
5
votes
2answers
105 views

Are on-line backpropagation iterations perpendicular to the constraint?

Raul Rojas' Neural Networks A Systematic Introduction, section 8.1.2 relates off-line backpropagation and on-line backpropagation with Gauss-Jacobi and Gauss-Seidel methods for finding the ...
1
vote
1answer
39 views

Which local minima to choose according to the shape of the error surface?

The following plot shows error function output based on system weights. Two equal local minima are shown in green pointers. Note that the red dots are not related to the question. Considering the ...
3
votes
2answers
60 views

Could error surface shape be useful to detect which local minima is better for generalization?

The following plot shows error function output based on system weights. Two equal local minima are shown in green pointers. Note that the red dots are not related to the question. Does the right one ...
2
votes
1answer
73 views

What is the gradient of the objective function in the Soft Actor-Critic paper?

In the paper "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor", they define the loss function for the policy network as $$ J_\pi(\phi)=\mathbb E_{...
1
vote
1answer
67 views

How to obtain a formula for loss, when given an iterative update rule in gradient descent?

From the reinforcement learning book section 13.3: Using pytorch, I need to calculate a loss, and then the gradient is calculated internally. How to obtain the loss from equations which are stated ...
0
votes
1answer
27 views

How to shape the weights or nodes during gradient training of neural network? Training with constraints?

Gradient training changes indiscriminately all the weights and nodes of the neural network. But one can imagine the situations when the training should be shaped, e.g.: One can put constraints on ...
2
votes
1answer
38 views

Will LMS always be convex function? If yes, then why do we change it for neural networks?

In LMS(least mean square) since, we use a quadratic error function, and quadratic functions are generally parabola in (some convex like shape). I wonder whether that is the reason why we use least ...
1
vote
1answer
29 views

Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs?

I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure: ...
6
votes
3answers
160 views

How can a neural network learn when the derivative of the activation function is 0?

Imagine that I have an artificial neural network with a single hidden layer and that I am using ReLU as my activating function. If by change I initialize my bias and my weights in such a form that: $$ ...
0
votes
1answer
169 views

Can gradient descent training be used for nonsmooth loss functions?

I have non-smooth loss function - e.g. loss(x)=min(x, 0.5). Can gradient descent be used for training neural networks with such functions. Can gradient descent be used for fairly general, ...
1
vote
1answer
306 views

Cost function increasing with SGD

In Deep Learning by Goodfellow et al., I came across the following line on the chapter on Stochastic Gradient Descent (pg. 287): The main question is how to set $\epsilon_0$. If it is too large, ...
1
vote
0answers
27 views

neuralnetworksanddeeplearning.com chapter 5 problems

For http://neuralnetworksanddeeplearning.com/chap5.html , could anyone suggest: 1) how to approach the derivation of expression (123) ? 2) what constitutes value ~ 0.45 ? 3) why the need of taylor ...
0
votes
1answer
293 views

Feed forward neural network using numpy for IRIS dataset

I tried to build a neural network for working on IRIS dataset using only numpy after reading an article (link: https://iamtrask.github.io/2015/07/12/basic-python-network/). I tried to search the ...
1
vote
2answers
254 views

SARSA won't work for linear function approximator for MountainCar-v0 in OpenAI environment. What are the possible causes?

I am learning Reinforcement Learning from the lectures from David Silver. I finished lecture 6 and went on to try SARSA with linear function approximator for MountainCar-v0 environment from OpenAI. A ...
2
votes
2answers
2k views

Can the mean squared error be negative?

I'm new to machine learning. I was watching a prof. Ng's video about gradient descent from the machine learning course. It said that we want our cost function (in this case, the mean squared error) to ...
1
vote
0answers
29 views

Neural network backpropagation gradient descent better than conjugate gradient descent?

My understanding is that the conjugate gradient method is faster than gradient descent because it does less zig zags while descending. How come the state of the art papers I see all use gradient ...
3
votes
1answer
289 views

Deep Q-Learning: why don't we use mini-batches during experience reply?

In examples and tutorial about DQN, I've often noticed that during the experience replay (training) phase people tend to use stochastic gradient descent / online learning. (e.g. link1, link2) ...
4
votes
4answers
3k views

How do I choose the optimal batch size?

Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size can be one of three options: batch mode: where the batch ...
2
votes
2answers
720 views

Should the weights of a neural network be updated after each example or at the end of the batch? [duplicate]

Should the weights of a neural network be updated after each example or at the end of the batch? Do I need a normalization factor in the second case?
2
votes
1answer
75 views

Weight Normalization paper

I am trying to dissect paper about weight normalization: https://papers.nips.cc/paper/6114-weight-normalization-a-simple-reparameterization-to-accelerate-training-of-deep-neural-networks.pdf ...
4
votes
1answer
89 views

Do good approximations produce good gradients?

Let’s say I have a neural net doing classification and I’m doing stochastic gradient descent to train it. If I know that my current approximation is a decent approximation, can I conclude that my ...
5
votes
1answer
1k views

Can non-differentiable layer be used in a neural network, if it's not learned?

For example, AFAIK pooling layer in CNN is not differentiable, but it can be used because it's not learning. Is it always true?
1
vote
0answers
29 views

Has anyone investigated iteration awareness beyond RNN and LSTM?

This question considers the convergence of an artificial networks (MLPs, RNNs, LSTM nets, CNNs) over time or over the course of epochs made up of iterations through training examples. In this ...
1
vote
2answers
96 views

Mini-batch training and the gradient

I am reading a book that states, "As the mini-batch size increases, the gradient computed is closer to the 'true' gradient". So basically what they are saying is mini-batch training only focuses on ...
0
votes
2answers
56 views

Behaviour of cost

In doing a project using neural networks with an input layer, 4 hidden layers and an output layer ,I used mini batch gradient descent. I noticed that the randomly initialised weights seemed to do a ...