7

There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include: Probabalistically accepting worse solutions in the hope that this will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing). Maintaining a list of recently-encountered states (or attributes ...


7

In general a cost function can be negative. The more negative, the better of course, because you are measuring a cost the objective is to minimise it. A standard Mean Squared Error function cannot be negative. The lowest possible value is $0$, when there is no output error from any example input. How can our cost function which is mean squared error have ...


7

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write it. $$(\hat{y} - y)^2 = (\hat{y} - y)(\hat{y} - y) = \hat{y}^2 -2\hat{y}y + y^2$$ $$(y - \hat{y})^2 = (y -\hat{y})(y - \hat{y}) = y^2 -2y\hat{y} + \hat{y}^2$$ ...


7

I'm going to use slightly different notation, $\leftarrow$ for an assignment, $\alpha$ for learning rate, $\nabla_w J$ in place of $g$* and implied multiplication as these are slightly more common. Also, using bold letters to represent vectors. In that notation, the update rule for basic gradient descent would be written as: $$\mathbf{w} \leftarrow \mathbf{...


6

There are several different algorithms that can be used for gradient free neural network training. Some of these algorithms include particle swarm optimization, genetic algorithms, simulated annealing, and several others. Almost any optimization algorithm can be used to train a neural network. Here is an overview of some of the algorithms I listed: Particle ...


6

It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function is differentiable*, and usually trivially so. For example: If an average pooling layer has inputs $z$ and outputs $a$, and each output is average of 4 inputs then $\frac{da}{dz} = 0.25$ (if pooling layers overlap it gets a ...


6

The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is \begin{align} \frac{d}{d \hat{y}} \mathcal{L_1} &= \frac{d}{d \hat{y}} (\hat{y} - y)^2 \\ &= 2(\hat{y} - y) \frac{d}{d \hat{y}} (\hat{y} - y) \\ &= 2(\hat{y} - y) (1) \\ &...


5

In general $|f(x) - f_k(x)| \leq \epsilon$ doesn't ensure $|\nabla f(x) - \nabla f_k(x)| \leq c\epsilon$. And for neural networks there is no reason to believe it will happen either. You can also look at Sobolev Training Paper (https://arxiv.org/abs/1706.04859). In particular, note that Sobolev training was better than critic training, which indirectly may ...


5

$g(x) = x^2$ is indeed a parabola and thus has just one optimum. However, the $\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i (y_i - f(x_i))^2$, where $\boldsymbol{x}$ are the inputs, $\boldsymbol{y}$ the corresponding labels and the function $f$ is the model (e.g. a neural network), is not necessarily a parabola. In general, it is only a parabola if $...


5

Calculation of gradient \begin{align} \nabla_{\theta} \log(\pi_{\theta}(a|s)) &= \phi(s,a) - \mathbb E[\phi (s, \cdot)]\\ &= \phi(s,a) - \sum_{a'} \pi(a'|s) \phi(s,a') \end{align} is only valid for linear function approximation with action preferences of form \begin{equation} h(s, a, \theta) = \theta^T \phi(s,a) \end{equation} and softmax policy \...


4

I think Musk was using the terminology correctly though perhaps with hyperbole. I believe this was tweeted in the context of the botnet attacks on name-resolution services that broke Netflix and a large number of other internet services for a time. He was expressing the idea that you could train a botnet-based system to attack the internet by giving it a ...


4

DQN "library" implementations that I have seen do use mini-batches to train, and I would generally recommend this, as it usually strikes a reasonable balance between number of weight updates and accuracy of the gradients. In your first link, and the code excerpt, the sample list is literally called minibatch. However, the developer then goes on to make a ...


4

Usually, when talking about regularization for neural networks there are 3 main types: L1, L2 and dropout. All affect the gradient descent procedure. L1 and L2 regularization is implemented in the loss function, and therefore are part of gradient descent directly by altering the derivatives of the loss function thereby altering the weight update rules of ...


3

If gradients are noisy (does this means that in some dimension we have small and in some high curvature or that error noise differs for very similar values of w?) Gradients being noisy means that they are "inconsistent" across different epochs / training steps. With that I mean that they'll sometimes point in one direction, later in a different (maybe the ...


3

This is standard backpropagation. The gradient term you see is in fact a vector of partial derivatives where each element is the partial derivative of the log-likelihood with respect to each element of the parameter vector $\theta$. Therefore, it has the same dimensionality as $\theta$. Each element of the parameter vector is then updated with the respective ...


3

Backpropagation with stride > 1 involves dilation of the gradient tensor with stride-1 zeroes. I created a blog post that describes this in greater detail.


3

There are 3 separate issues that are often confounded in Deep Learning and Neuroscience: Deep Learning is inspired by the way the biological brain works. Deep Learning is how the biological brain works. Deep Learning can model how the biological brain works. Number 1 is accurate. The brain has many layers and many connections. Those principles have ...


3

There is not single answer to the vanishing gradient problem. However, there a few things that can help. As mentioned in the comments, use of Rectified Linear Units (ReLU) as your activation function can help, since the it does not get saturated for large neuron inputs. Next, careful choice of weight initialization can help avoid saturation, as well. See ...


3

In a setup like the above where the derivat[iv]e is 0 is it true that an NN won´t learn anything? There are a couple of adjustments to gradients that might apply if you do this in a standard framework: Momentum may cause weights to continue changing if any recent ones were non-zero. This is typically implemented as a rolling mean of recent gradients. ...


3

In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations: what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima looking at the LF steepness we observe the Left LM is in a steeper region than ...


3

NEAT is a genetic algorithm (GA). A genetic algorithm maintains a population of individuals (or chromosomes) and evolves it using operations like the crossover or the mutation, so that the fittest individuals keep living and most other individuals die. The nature of the individuals depends on the problem. For example, in the case of NEAT, the individuals are ...


3

The paper The Comparison and Combination of Genetic and Gradient Descent Learning in Recurrent Neural Networks: An Application to Speech Phoneme Classification (2007), by Rohitash Chandra and Christian W. Omlin, uses genetic algorithms to train a recurrent neural network and then uses gradient descent to fine tune the trained model. The paper Evolutionary ...


3

Welcome to AI Stack exchange! You're right, as the network is initialised randomly, the resultant function is essentially impossible to get your head around. This is because most of the time the network has >4 dimensions (4 can be graphed with some effort and a lot of color), and as such is literally beyond human comprehension via graphing. So what do we ...


3

I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that function IS. It's usually called the loss function (and, in general, objective function) and often denoted as $\mathcal{L}$ or $L$ (or something like that, i.e. it is not really important how you denote it). The specific function used as a ...


3

The book Deep Learning by Goodfellow, Bengio, and Courville says (Sec 8.3.3, p 292 in my copy) states that Unfortunately, in the stochastic gradient case, Nesterov momentum does not improve the rate of convergence. I'm not sure why this is, but the theoretical advantage depends on a convex problem, and from this, it sounds like the practical advantage ...


3

Introduction First of all, it's completely normal that you are confused because nobody really explains this well and accurately enough. Here's my partial attempt to do that. So, this answer doesn't completely answer the original question. In fact, I leave some unanswered questions at the end (that I will eventually answer). The gradient is a linear ...


2

What you are describing is conceptually close to adversarial training. you should read more on adversarial examples and generative adversarial networks for more information. The idea is that there is a discriminator network, whose job is to correctly discriminate between positive and negative examples. We also have a generative network, that learns to ...


2

For a function of one variable, there are only two options for directions in the domain: left or right, so it becomes almost trivial, but you can still talk about gradient descent. You would take steps to the left if the slope/derivative is positive and make steps to the right if the slope/derivative is negative--i.e. the opposite direction of the ...


2

That is similar to a chicken-and-egg problem; If the second network could figure out the direction and learn faster, why couldn't the first network do that too and learn even faster. However, I would recommend looking up Actor Critic Methods. Here there are two networks but one (the critic) evaluates the other (the actor) to head it in the right direction.


2

Don’t think about it as the being proportional to something. Think about it this way: I’m now at . Where do I want to be at Time step so that the error decreases? For that, I need to know how the error changes when I make small steps to the left or right of If increases as I increase (that is, if , then obviously, I would want to move a little bit to ...


Only top voted, non community-wiki answers of a minimum length are eligible