People who code: we want your input. Take the Survey

For questions surrounding gradient descent, a method for finding the optimum state of a parameterized function based on another function often called the loss or error function. It iteratively descends the loss surface to the minimum loss by adjusting parameters based on the product of the partial derivatives comprising the gradient and a learning rate.

156 questions
Filter by
Sorted by
Tagged with
142 views

### How long should the state-dependent baseline for policy gradient methods be trained at each iteration?

How long should the state-dependent baseline be trained at each iteration? Or what baseline loss should we target at each iteration for use with policy gradient methods? I'm using this equation to ...
826 views

Semi-gradient methods work well in reinforcement learning, but what is there a reason of not using the true gradient if it can be computed? I tried it on the cart pole problem with a deep Q-network ...
46 views

### RMSprop equation - dividing by a matrix?

I've been trying to understand RMSprop for a long time, but there's something that keeps eluding me. $dW$ and $db$ are matrices (that's what I understand from the element-wise comment), so that must ...
232 views

It (Adagrad) adapts the learning rate to the parameters, performing smaller updates (i.e. low learning rates) for parameters associated with frequently occurring features, and larger updates (i.e. ...
108 views

### During neural network training, can gradients leak sensitive information in case training data fed is encrypted (homomorphic)?

Some algorithms in the literature allow recovering the input data used to train a neural network. This is done using the gradients (updates) of weights, such as in Deep Leakage from Gradients (2019) ...
2k views

### How to perform gradient checking in a neural network with batch normalization?

I have implemented a neural network (NN) using python and numpy only for learning purposes. I have already coded learning rate, momentum, and L1/L2 regularization and checked the implementation with ...
65 views

### Why is loss displayed as a parabola in mean squared error with gradient descent?

I'm looking at the loss function: mean squared error with gradient descent in machine learning. I'm building a single-neuron network (perceptron) that outputs a linear number. For example: Input * ...
141 views

### How to obtain a formula for loss, when given an iterative update rule in gradient descent?

From the reinforcement learning book section 13.3: Using pytorch, I need to calculate a loss, and then the gradient is calculated internally. How to obtain the loss from equations which are stated ...
30 views

### How can the gradient of the weight be calculated in the viewpoint of matrix calculus?

Let $\sigma(x)$ be sigmoid function. Consider the case where $\text{out}=\sigma(\vec{x} \times W + \vec{b})$, and we want to compute $\frac{\partial{\text{out}}}{\partial{w} }.$ Set the dimension as ...
31 views

In the context of a neural network $\hat{y} = f_\theta(\mathbf{x})$ with parameters $\theta$ that is trained to perform regression such that the prediction $\hat{\mathbf{y}} = [\hat{y}_1,\hat{y}_2,...,... 3answers 205 views ### Does gradient descent in deep learning assume a smooth fitness landscape? I've come across the concept of fitness landscape before and, in my understanding, a smooth fitness landscape is one where the algorithm can converge on the global optimum through incremental ... 1answer 125 views ### How do you perform a gradient based adversarial attack on an SVM based model? I have an SVM currently and want to perform a gradient based attack on it similar to FGSM discussed in Explaining And Harnessing Adversarial Examples. I am struggling to actually calculate the ... 1answer 133 views ### What is the purpose of argmax in the PPO algorithm? I'm kinda new to machine learning and still not too solid on math and particularly calculus. I'm currently trying to implement PPO algorithm as described in the spiningUp website : This line is ... 0answers 34 views ### Optimizer that prevents parameters from oscillating When we perform gradient descent, especially in an online setting where the training data is presented in a non-random order, a particular 1-dimensional parameter (such as an edge weight) may first ... 1answer 258 views ### What is the impact of scaling the KL divergence and reconstruction loss in the VAE objective function? Variational autoencoders have two components in their loss function. The first component is the reconstruction loss, which for image data, is the pixel-wise difference between the input image and ... 2answers 157 views ### Why the cost/loss starts to increase for some iterations during the training phase? I am trying to build a recurrent neural network from scratch. It's a very simple model. I am trying to train it to predict two words (dogs and gods). While training, the value of cost function starts ... 0answers 17 views ### What is the effect and behavior of using mixed weight instead of normal weight matrix? Suppose I try to find appropriate matrix A in differential equation$\dot{X}=A X$using RNN. Current state is$X=\begin{bmatrix} x_{1}\\ x_{2}\\ \end{bmatrix}$, and desired trajectory state is$X_d=\...
24 views

I am troubled by natural gradient methods. If we have a function f(x) we wish to minimize, gradient descent minimizes f(x) of course, but what does the natural gradient do? I found on https://...
68 views

### In gradient descent's update rule, why do we use $\sigma(z^{l-1})\frac{\delta C_0}{ \delta w^{l}}$ instead of $\frac{\delta C_0}{\delta w^{l}}$?

I am trying to code a two layered neural network simple NN as I have described here https://itisexplained.com/html/NN/ml/5_codingneuralnetwork/ I am getting stuck on the last step of updating the ...
24 views

### How to deal with losses on different scales in multi-task learning?

Say I'm training a model for multiple tasks by trying to minimize sum of losses $L_1 + L_2$ via gradient descent. If these losses are on a different scale, the one whose range is greater will dominate ...
26 views

### What is the name of this algorithm that estimates the gradient with an average by sampling from a distribution?

Consider maximizing the function $R(w)$ with parameter $w$ using gradient ascent. However, we don't know the gradient $\nabla_wR(w)$ formula. Now suppose $w$ is sampled from a probability distribution ...
450 views

### What is the advantage of using cross entropy loss & softmax?

I am trying to do the standard MNIST dataset image recognition test with a standard feed forward NN, but my network failed pretty badly. Now I have debugged it quite a lot and found & fixed some ...
66 views

### Is batch learning with gradient descent equivalent to “rehearsal” in incremental learning?

I am learning about incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? ...
74 views

### Bias gradient of layer before batch normalization always zero

From the original paper and this post we have that batch normalization backpropagation can be formulated as I'm interested in the derivative of the previous layer outputs $x_i=\sigma(w X_i+b)$ with ...
325 views

### Is there an ideal range of learning rate which always gives a good result almost in all problems?

I once read somewhere that there is a range of learning rate within which learning is optimal in almost all the cases, but I can't find any literature about it. All I could get is the following graph ...
33 views

### Should I use batch gradient descent when I have a small sample size?

I have a dataset with an input size of 155x155, with the output being 155 x 1 with a 3-4 layer neural net being used for regression. With such a small sample size, should I use full batch gradient ...
169 views

### Why is the perceptron criterion function differentiable?

I'm reading chapter one of the book called Neural Networks and Deep Learning from Aggarwal. In section 1.2.1.1 of the book, I'm learning about the perceptron. One thing that book says is, if we use ...
57 views

### Why are most commonly used activation functions continuous?

I have come to notice that the most commonly used activation functions are continuous. Is there any specific reason behind this? Results such as this paper have worked on training networks with ...
23 views

### Is it possible to ensure the convergence when training a RNN weight on its SVD decomposition?

I'm reading the following paper in which the author seems to do 2 things interesting: The hidden-to-hidden weight matrix of the RNN is SVD decomposed and train separately. Each orthogonal part of the ...
137 views

### Why evolutionary training of neural networks is not popular?

Evolutionary algorithms are mentioned in some sources as a method to train a neural network (finding weights, not hyperparameters). However, I have not heard about one practical application of such an ...
1k views

### How is the gradient of the loss function in DQN derived?

In the original DQN paper, page 1, the loss function of the DQN is $$L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim U(D)} [(r+\gamma \max_{a'} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}))^2]$$ ...
292 views

### Understanding the derivation of the first-order model-agnostic meta-learning

According to the authors of this paper, to improve the performance, they decided to drop backward pass and using a first-order approximation I found a blog which discussed how to derive the math ...
32 views

### Backpropagation implementation not applicable for other cases

I saw this implementation of backpropagation in MATLAB, where the loss function used is MSE, and the last layer's activation function was sigmoid. I denoted the portions of the formula for what I ...
43 views

35 views

### How to use unmodified input in neural network?

My question may be a bit hard to explain... My neural network learns a categorical distribution, which serves as an index. This index will look up the value (= action_mean) in Input 2. From this ...
I am new to Deep Learning. Suppose that we have a neural network with one input layer, one output layer, and one hidden layer. Let's refer to the weights from input to hidden as $W$ and the weights ...