11

Should I be changing the weights/biases on every single sample before moving on to the next sample, You can do this, it is called stochastic gradient descent (SGD) and typically you will shuffle the dataset before working through it each time. or should I first calculate the desired changes for the entire lot of 1,000 samples, and only then start ...


8

There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include: Probabalistically accepting worse solutions in the hope that this will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing). Maintaining a list of recently-encountered states (or attributes ...


8

In general a cost function can be negative. The more negative, the better of course, because you are measuring a cost the objective is to minimise it. A standard Mean Squared Error function cannot be negative. The lowest possible value is $0$, when there is no output error from any example input. How can our cost function which is mean squared error have ...


7

It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function is differentiable*, and usually trivially so. For example: If an average pooling layer has inputs $z$ and outputs $a$, and each output is average of 4 inputs then $\frac{da}{dz} = 0.25$ (if pooling layers overlap it gets a ...


7

The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is \begin{align} \frac{d}{d \hat{y}} \mathcal{L_1} &= \frac{d}{d \hat{y}} (\hat{y} - y)^2 \\ &= 2(\hat{y} - y) \frac{d}{d \hat{y}} (\hat{y} - y) \\ &= 2(\hat{y} - y) (1) \\ &...


7

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write it. $$(\hat{y} - y)^2 = (\hat{y} - y)(\hat{y} - y) = \hat{y}^2 -2\hat{y}y + y^2$$ $$(y - \hat{y})^2 = (y -\hat{y})(y - \hat{y}) = y^2 -2y\hat{y} + \hat{y}^2$$ ...


7

I'm going to use slightly different notation, $\leftarrow$ for an assignment, $\alpha$ for learning rate, $\nabla_w J$ in place of $g$* and implied multiplication as these are slightly more common. Also, using bold letters to represent vectors. In that notation, the update rule for basic gradient descent would be written as: $$\mathbf{w} \leftarrow \mathbf{...


7

$\max(-y_i(w x_i), 0)$ is not partial derivable respect $w$ if $w x_i=0$. Loss functions are problematic when not derivable in some point, but even more when they are flat (constant) in some interval of the weights. Assume $y_i = 1$ and $w x_i < 0$ (that is, an error of type "false negative"). In this case, function $[y_i - \text{sign}(w x_i)]^2 ...


6

There are several different algorithms that can be used for gradient free neural network training. Some of these algorithms include particle swarm optimization, genetic algorithms, simulated annealing, and several others. Almost any optimization algorithm can be used to train a neural network. Here is an overview of some of the algorithms I listed: Particle ...


6

Short answers Is back-propagation applied immediately after getting the output for each input or after getting the output for all inputs in a batch? You can perform back-propagation using (or after) only one training input (also known as data point, example, sample or observation) or multiple ones (a batch). However, the loss function to train the neural ...


5

I think Musk was using the terminology correctly though perhaps with hyperbole. I believe this was tweeted in the context of the botnet attacks on name-resolution services that broke Netflix and a large number of other internet services for a time. He was expressing the idea that you could train a botnet-based system to attack the internet by giving it a ...


5

In general $|f(x) - f_k(x)| \leq \epsilon$ doesn't ensure $|\nabla f(x) - \nabla f_k(x)| \leq c\epsilon$. And for neural networks there is no reason to believe it will happen either. You can also look at Sobolev Training Paper (https://arxiv.org/abs/1706.04859). In particular, note that Sobolev training was better than critic training, which indirectly may ...


5

DQN "library" implementations that I have seen do use mini-batches to train, and I would generally recommend this, as it usually strikes a reasonable balance between number of weight updates and accuracy of the gradients. In your first link, and the code excerpt, the sample list is literally called minibatch. However, the developer then goes on to make a ...


5

$g(x) = x^2$ is indeed a parabola and thus has just one optimum. However, the $\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i (y_i - f(x_i))^2$, where $\boldsymbol{x}$ are the inputs, $\boldsymbol{y}$ the corresponding labels and the function $f$ is the model (e.g. a neural network), is not necessarily a parabola. In general, it is only a parabola if $...


5

Calculation of gradient \begin{align} \nabla_{\theta} \log(\pi_{\theta}(a|s)) &= \phi(s,a) - \mathbb E[\phi (s, \cdot)]\\ &= \phi(s,a) - \sum_{a'} \pi(a'|s) \phi(s,a') \end{align} is only valid for linear function approximation with action preferences of form \begin{equation} h(s, a, \theta) = \theta^T \phi(s,a) \end{equation} and softmax policy \...


5

Introduction First of all, it's completely normal that you are confused because nobody really explains this well and accurately enough. Here's my partial attempt to do that. So, this answer doesn't completely answer the original question. In fact, I leave some unanswered questions at the end (that I will eventually answer). The gradient is a linear ...


5

There isn't any explicit relation between the batch size and the gradient accumulation steps, except for the fact that gradient accumulation helps one to fit models with relatively larger batch sizes (typically in single-GPU setups) by cleverly avoiding memory issues. The core idea of gradient accumulation is to perform multiple backward passes using the ...


5

If the learning rate is greater than or equal to $1$ the Robbins-Monro condition $$\sum _{{t=0}}^{{\infty }}a_{t}^{2}<\infty\label{1}\tag{1},$$ where $a_t$ is the learning rate at iteration $t$, does not hold (given that a number bigger than $1$ squared becomes a bigger number), so stochastic gradient descent is not generally guaranteed to converge to a ...


5

The 2015 article Cyclical Learning Rates for Training Neural Networks by Leslie N. Smith gives some good suggestions for finding an ideal range for the learning rate. The paper's primary focus is the benefit of using a learning rate schedule that varies learning rate cyclically between some lower and upper bound, instead of trying to choose a single fixed ...


4

In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations: what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima looking at the LF steepness we observe the Left LM is in a steeper region than ...


4

What are filters in image processing? In the context of image processing (and, in general, signal processing), the kernels (also known as filters) are used to perform some specific operation on the image. For example, you can use a Gaussian filter to smooth the image (including its edges). What are filters in CNNs? In the context of convolutional neural ...


4

The paper The Comparison and Combination of Genetic and Gradient Descent Learning in Recurrent Neural Networks: An Application to Speech Phoneme Classification (2007), by Rohitash Chandra and Christian W. Omlin, uses genetic algorithms to train a recurrent neural network and then uses gradient descent to fine tune the trained model. The paper Evolutionary ...


4

The main evolutionary algorithm used to train neural networks is Neuro-Evolution of Augmenting Topoloigies, or NEAT. NEAT has seen fairly widespread use. There are thousands of academic papers building on or using the algorithm. NEAT is not widely used in commercial applications because if you have a clean objective function, a topology that is optimized ...


4

The book Deep Learning by Goodfellow, Bengio, and Courville says (Sec 8.3.3, p 292 in my copy) states that Unfortunately, in the stochastic gradient case, Nesterov momentum does not improve the rate of convergence. I'm not sure why this is, but the theoretical advantage depends on a convex problem, and from this, it sounds like the practical advantage ...


4

Usually, when talking about regularization for neural networks there are 3 main types: L1, L2 and dropout. All affect the gradient descent procedure. L1 and L2 regularization is implemented in the loss function, and therefore are part of gradient descent directly by altering the derivatives of the loss function thereby altering the weight update rules of ...


4

The visualisation can be found in The need for small learning rates on large problems. This paper by D. Randall Wilson and Tony R. Martinez from 2001 investigates the role of learning rates in gradient descent algorithms. In general, different algorithms assign different meaning to the same word 'learning rate'. For example, the learning rate in a gradient ...


3

This is standard backpropagation. The gradient term you see is in fact a vector of partial derivatives where each element is the partial derivative of the log-likelihood with respect to each element of the parameter vector $\theta$. Therefore, it has the same dimensionality as $\theta$. Each element of the parameter vector is then updated with the respective ...


3

Backpropagation with stride > 1 involves dilation of the gradient tensor with stride-1 zeroes. I created a blog post that describes this in greater detail.


3

There are 3 separate issues that are often confounded in Deep Learning and Neuroscience: Deep Learning is inspired by the way the biological brain works. Deep Learning is how the biological brain works. Deep Learning can model how the biological brain works. Number 1 is accurate. The brain has many layers and many connections. Those principles have ...


Only top voted, non community-wiki answers of a minimum length are eligible