Questions tagged [gradient-descent]

For questions surrounding gradient descent, a method for finding the optimum state of a parameterized function based on another function often called the loss or error function. It iteratively descends the loss surface to the minimum loss by adjusting parameters based on the product of the partial derivatives comprising the gradient and a learning rate.

Filter by
Sorted by
Tagged with
5
votes
2answers
90 views

Is the gradient at a layer independent of the activations of the previous layers?

Is the gradient at a layer (of a feed-forward neural network) independent of the activations of the previous layers? I read this in a paper titled Mean Field Residual Networks: On the Edge of Chaos (...
0
votes
1answer
40 views

Backpropagation: Chain Rule to the Third Last Layer

I'm trying to solve dLoss/dW1. The network is as in picture below with identity activation at all neurons: Solving dLoss/dW7 is simple as there's only 1 way to output: $Delta = Out-Y$ $Loss = abs(...
2
votes
1answer
58 views

How to calculate multiobjective optimization cost for ordinary problems?

What I did: Created a population of 2D legged robots in a simulated environment. Found the best motor rotation values to make the robots move rightward, using an objective function with Differential ...
4
votes
1answer
108 views

Why is batch gradient descent performing worse than stochastic and minibatch gradient descent?

I have implemented a neural network from scratch (only using numpy) and I am having problems understanding why the results are so different between stochastic/minibatch gradient descent and batch ...
5
votes
1answer
149 views

Why evolutionary training of neural networks is not popular?

Evolutionary algorithms are mentioned in some sources as a method to train a neural network (finding weights, not hyperparameters). However, I have not heard about one practical application of such an ...
0
votes
1answer
341 views

Network doesn't converge with ReLU or Leaky ReLU, but works well with sigmoid/tanh

I have these training data to separate, the classes are rather randomly scattered: My first attempt was using tf.nn.relu activation function, but output was stuck ...
4
votes
1answer
1k views

How is the gradient of the loss function in DQN derived?

In the original DQN paper, page 1, the loss function of the DQN is $$ L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim U(D)} [(r+\gamma \max_{a'} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}))^2] $$ ...
5
votes
1answer
253 views

An intuitive explanation of Adagrad, its purpose and its formula

It (Adagrad) adapts the learning rate to the parameters, performing smaller updates (i.e. low learning rates) for parameters associated with frequently occurring features, and larger updates (i.e. ...
3
votes
0answers
84 views

Does Retina-net's focal loss accomplish its goal?

Taking out the weighting factor we can define focal loss as $$FL(p) = -(1-p)^\gamma log(p) $$ Where $p$ is the target probability. The idea being that single stage object detectors have a huge ...
0
votes
1answer
88 views

Is it possible with stochastic gradient descent for the error to increase?

As simple as that. Is there any scenario where the error might increase, if only by a tiny amount, when using SGD (no momentum)?
6
votes
2answers
583 views

Can neuroevolution be combined with gradient descent?

Is there any precedent for using a neuroevolution algorithm, like NEAT, as a way of getting to an initialization of weights for a network that can then be fine-tuned with gradient descent and back-...
1
vote
1answer
112 views

In NN, as iterations of Gradient descent increases, the accuracy of Test/CV set decreases. how can i resolve this?

As mentioned in the title I'm using 300 Dataset example with 500 feature as an input. As I'm training the dataset, I found something peculiar. Please look at the data shown below. Iteration 5000 |...
2
votes
0answers
58 views

Neural networks when gradient descent is not possible

I am looking for an example in which it is simply impossible to use some sort of gradient descent to train a neural network. Is this available? I have read quite some papers about gradient-free ...
1
vote
1answer
104 views

How does NEAT find the most successful generation without gradients?

I'm new to NEAT, so, please, don't be too harsh. How does NEAT find the most successful generation without gradient descent or gradients?
5
votes
3answers
206 views

Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The MSE can be defined as $(\hat{y} - y)^2$, which should be equal to $(y - \hat{y})^2$, but I think their derivative is different, so I am confused of what derivative will I use for computing my ...
10
votes
2answers
909 views

Is neural networks training done one-by-one? [duplicate]

I'm trying to learn neural networks by watching this series of videos and implementing a simple neural network in Python. Here's one of the things I'm wondering about: I'm training the neural network ...
3
votes
2answers
152 views

Are filters fixed or learned?

No matter what I google or what paper I read, I can't find an answer to my question. In a deep convolutional neural network, let's say AlexNet (Krizhevsky, 2012), filters' weights are learned by means ...
3
votes
0answers
486 views

What is the right formula for weight update rule in Logistic Regression using stochastic gradient descent

Apologies for the lengthy title. My question is about the weight update rule for logistic regression using stochastic gradient descent. I have just started experimenting on Logistic Regression. I ...
6
votes
2answers
1k views

How is local minima possible in gradient descent?

Gradient descent works on the equation of mean squared error, which is an equation of a parabola $y=x^2$. We often say that weight adjustment in a neural network by gradient descent algorithm can hit ...
2
votes
1answer
79 views

How can we reach global optimum?

Gradient descent can get stuck into local optimum. Which techniques are there to reach global optimum?
1
vote
1answer
326 views

When is bias values updated in back propagation?

I am new to deep learning. I have doubts on modifying bias values during back propagation. My doubts are Does the back propagation algorithm modifies the weigh values and bias values in the same pass?...
4
votes
1answer
2k views

Is back-propagation applied for each data point or for a batch of data points?

I am new to deep learning and trying to understand the concept of back-propagation. I have a doubt about when the back-propagation is applied. Assume that I have a training data set of 1000 images ...
2
votes
1answer
1k views

Does backpropagation update weights one layer at a time?

I am new to Deep Learning. Suppose that we have a neural network with one input layer, one output layer, and one hidden layer. Let's refer to the weights from input to hidden as $W$ and the weights ...
5
votes
2answers
114 views

Are on-line backpropagation iterations perpendicular to the constraint?

Raul Rojas' Neural Networks A Systematic Introduction, section 8.1.2 relates off-line backpropagation and on-line backpropagation with Gauss-Jacobi and Gauss-Seidel methods for finding the ...
1
vote
1answer
54 views

Which local minima to choose according to the shape of the error surface?

The following plot shows error function output based on system weights. Two equal local minima are shown in green pointers. Note that the red dots are not related to the question. Considering the ...
4
votes
2answers
71 views

Could error surface shape be useful to detect which local minima is better for generalization?

The following plot shows error function output based on system weights. Two equal local minima are shown in green pointers. Note that the red dots are not related to the question. Does the right one ...
3
votes
2answers
254 views

What is the gradient of the objective function in the Soft Actor-Critic paper?

In the paper Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, they define the loss function for the policy network as $$ J_\pi(\phi)=\mathbb E_{s_t\...
3
votes
2answers
152 views

How to obtain a formula for loss, when given an iterative update rule in gradient descent?

From the reinforcement learning book section 13.3: Using pytorch, I need to calculate a loss, and then the gradient is calculated internally. How to obtain the loss from equations which are stated ...
0
votes
1answer
44 views

How to shape the weights or nodes during gradient training of neural network? Training with constraints?

Gradient training changes indiscriminately all the weights and nodes of the neural network. But one can imagine the situations when the training should be shaped, e.g.: One can put constraints on ...
2
votes
1answer
75 views

Will LMS always be convex function? If yes, then why do we change it for neural networks?

In LMS(least mean square) since, we use a quadratic error function, and quadratic functions are generally parabola in (some convex like shape). I wonder whether that is the reason why we use least ...
1
vote
1answer
46 views

Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs?

I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure: ...
6
votes
3answers
469 views

How can a neural network learn when the derivative of the activation function is 0?

Imagine that I have an artificial neural network with a single hidden layer and that I am using ReLU as my activating function. If by change I initialize my bias and my weights in such a form that: $$ ...
1
vote
1answer
831 views

Can gradient descent training be used for non-smooth loss functions?

I have non-smooth loss function $f(x) = \min(x, 0.5)$. Can gradient descent be used for training neural networks with such functions? Can gradient descent be used for fairly general, mathematically ...
1
vote
1answer
1k views

Cost function increasing with SGD

In Deep Learning by Goodfellow et al., I came across the following line on the chapter on Stochastic Gradient Descent (pg. 287): The main question is how to set $\epsilon_0$. If it is too large, ...
1
vote
0answers
32 views

neuralnetworksanddeeplearning.com chapter 5 problems

For http://neuralnetworksanddeeplearning.com/chap5.html , could anyone suggest: 1) how to approach the derivation of expression (123) ? 2) what constitutes value ~ 0.45 ? 3) why the need of taylor ...
2
votes
1answer
725 views

Feed forward neural network using numpy for IRIS dataset

I tried to build a neural network for working on IRIS dataset using only numpy after reading an article (link: https://iamtrask.github.io/2015/07/12/basic-python-network/). I tried to search the ...
1
vote
2answers
636 views

SARSA won't work for linear function approximator for MountainCar-v0 in OpenAI environment. What are the possible causes?

I am learning Reinforcement Learning from the lectures from David Silver. I finished lecture 6 and went on to try SARSA with linear function approximator for MountainCar-v0 environment from OpenAI. A ...
6
votes
3answers
5k views

Can the mean squared error be negative?

I'm new to machine learning. I was watching a Prof. Andrew Ng's video about gradient descent from the machine learning online course. It said that we want our cost function (in this case, the mean ...
1
vote
0answers
32 views

Neural network backpropagation gradient descent better than conjugate gradient descent?

My understanding is that the conjugate gradient method is faster than gradient descent because it does less zig zags while descending. How come the state of the art papers I see all use gradient ...
4
votes
1answer
1k views

Deep Q-Learning: why don't we use mini-batches during experience reply?

In examples and tutorial about DQN, I've often noticed that during the experience replay (training) phase people tend to use stochastic gradient descent / online learning. (e.g. link1, link2) ...
2
votes
2answers
1k views

Should the weights of a neural network be updated after each example or at the end of the batch? [duplicate]

Should the weights of a neural network be updated after each example or at the end of the batch? Do I need a normalization factor in the second case?
3
votes
1answer
3k views

How do I calculate the gradient of the hinge loss function?

With reference to the research paper entitled Sentiment Embeddings with Applications to Sentiment Analysis, I am trying to implement its sentiment ranking model in Python, for which I am required to ...
4
votes
1answer
106 views

Do good approximations produce good gradients?

Let’s say I have a neural net doing classification and I’m doing stochastic gradient descent to train it. If I know that my current approximation is a decent approximation, can I conclude that my ...
9
votes
1answer
3k views

Can non-differentiable layer be used in a neural network, if it's not learned?

For example, AFAIK pooling layer in CNN is not differentiable, but it can be used because it's not learning. Is it always true?
2
votes
2answers
127 views

What's the rationale behind mini-batch gradient descent?

I am reading a book that states As the mini-batch size increases, the gradient computed is closer to the 'true' gradient So, I assume that they are saying that mini-batch training only focuses on ...
-1
votes
2answers
78 views

Behaviour of cost

In doing a project using neural networks with an input layer, 4 hidden layers and an output layer ,I used mini batch gradient descent. I noticed that the randomly initialised weights seemed to do a ...
5
votes
1answer
275 views

For each epoch, can I use only on a subset of the full training dataset to train the neural network?

If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini-batch gradient descent on this entire dataset is too high, can I ...
2
votes
1answer
380 views

How can we calculate the gradient of the Boltzmann policy over reward function?

I'm struggling with an inverse reinforcement learning problem which seems to appear quite often around the literature, yet I can't find any resources explaining it. The problem is that of calculating ...
3
votes
1answer
269 views

What does the notation $\nabla_\theta \mathcal{L}$ mean?

Here's the general algorithm of maximum entropy inverse reinforcement learning. This uses a gradient descent algorithm. The point that I do not understand is there is only a single gradient value $\...
3
votes
1answer
565 views

Why Feature Scaling for skewed contour?

Why is it that the skewed contour (unscaled features) will result in slow performance of gradient descent? In other words, how (or why) will the gradients end up taking a long time before finding the ...