7

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write it. $$(\hat{y} - y)^2 = (\hat{y} - y)(\hat{y} - y) = \hat{y}^2 -2\hat{y}y + y^2$$ $$(y - \hat{y})^2 = (y -\hat{y})(y - \hat{y}) = y^2 -2y\hat{y} + \hat{y}^2$$ ...


7

I'm going to use slightly different notation, $\leftarrow$ for an assignment, $\alpha$ for learning rate, $\nabla_w J$ in place of $g$* and implied multiplication as these are slightly more common. Also, using bold letters to represent vectors. In that notation, the update rule for basic gradient descent would be written as: $$\mathbf{w} \leftarrow \mathbf{...


6

There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include: Probabalistically accepting worse solutions in the hope that this will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing). Maintaining a list of recently-encountered states (or attributes ...


6

In general a cost function can be negative. The more negative, the better of course, because you are measuring a cost the objective is to minimise it. A standard Mean Squared Error function cannot be negative. The lowest possible value is $0$, when there is no output error from any example input. How can our cost function which is mean squared error have ...


6

The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is \begin{align} \frac{d}{d \hat{y}} \mathcal{L_1} &= \frac{d}{d \hat{y}} (\hat{y} - y)^2 \\ &= 2(\hat{y} - y) \frac{d}{d \hat{y}} (\hat{y} - y) \\ &= 2(\hat{y} - y) (1) \\ &...


5

In general $|f(x) - f_k(x)| \leq \epsilon$ doesn't ensure $|\nabla f(x) - \nabla f_k(x)| \leq c\epsilon$. And for neural networks there is no reason to believe it will happen either. You can also look at Sobolev Training Paper (https://arxiv.org/abs/1706.04859). In particular, note that Sobolev training was better than critic training, which indirectly may ...


5

$g(x) = x^2$ is indeed a parabola and thus has just one optimum. However, the $\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i (y_i - f(x_i))^2$, where $\boldsymbol{x}$ are the inputs, $\boldsymbol{y}$ the corresponding labels and the function $f$ is the model (e.g. a neural network), is not necessarily a parabola. In general, it is only a parabola if $...


5

Calculation of gradient \begin{align} \nabla_{\theta} \log(\pi_{\theta}(a|s)) &= \phi(s,a) - \mathbb E[\phi (s, \cdot)]\\ &= \phi(s,a) - \sum_{a'} \pi(a'|s) \phi(s,a') \end{align} is only valid for linear function approximation with action preferences of form \begin{equation} h(s, a, \theta) = \theta^T \phi(s,a) \end{equation} and softmax policy \...


4

I think Musk was using the terminology correctly though perhaps with hyperbole. I believe this was tweeted in the context of the botnet attacks on name-resolution services that broke Netflix and a large number of other internet services for a time. He was expressing the idea that you could train a botnet-based system to attack the internet by giving it a ...


4

It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function is differentiable*, and usually trivially so. For example: If an average pooling layer has inputs $z$ and outputs $a$, and each output is average of 4 inputs then $\frac{da}{dz} = 0.25$ (if pooling layers overlap it gets a ...


3

If gradients are noisy (does this means that in some dimension we have small and in some high curvature or that error noise differs for very similar values of w?) Gradients being noisy means that they are "inconsistent" across different epochs / training steps. With that I mean that they'll sometimes point in one direction, later in a different (maybe the ...


3

This is standard backpropagation. The gradient term you see is in fact a vector of partial derivatives where each element is the partial derivative of the log-likelihood with respect to each element of the parameter vector \theta. Therefore, it has the same dimensionality as \theta. Each element of the parameter vector is then updated with the respective ...


3

There are 3 separate issues that are often confounded in Deep Learning and Neuroscience: Deep Learning is inspired by the way the biological brain works. Deep Learning is how the biological brain works. Deep Learning can model how the biological brain works. Number 1 is accurate. The brain has many layers and many connections. Those principles have ...


3

There are several different algorithms that can be used for gradient free neural network training. Some of these algorithms include particle swarm optimization, genetic algorithms, simulated annealing, and several others. Almost any optimization algorithm can be used to train a neural network. Here is an overview of some of the algorithms I listed: Particle ...


3

There is not single answer to the vanishing gradient problem. However, there a few things that can help. As mentioned in the comments, use of Rectified Linear Units (ReLU) as your activation function can help, since the it does not get saturated for large neuron inputs. Next, careful choice of weight initialization can help avoid saturation, as well. See ...


3

DQN "library" implementations that I have seen do use mini-batches to train, and I would generally recommend this, as it usually strikes a reasonable balance between number of weight updates and accuracy of the gradients. In your first link, and the code excerpt, the sample list is literally called minibatch. However, the developer then goes on to make a ...


3

In a setup like the above where the derivat[iv]e is 0 is it true that an NN won´t learn anything? There are a couple of adjustments to gradients that might apply if you do this in a standard framework: Momentum may cause weights to continue changing if any recent ones were non-zero. This is typically implemented as a rolling mean of recent gradients. ...


3

In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations: what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima looking at the LF steepness we observe the Left LM is in a steeper region than ...


3

NEAT is a genetic algorithm (GA). A genetic algorithm maintains a population of individuals (or chromosomes) and evolves it using operations like the crossover or the mutation, so that the fittest individuals keep living and most other individuals die. The nature of the individuals depends on the problem. For example, in the case of NEAT, the individuals are ...


3

Welcome to AI Stack exchange! You're right, as the network is initialised randomly, the resultant function is essentially impossible to get your head around. This is because most of the time the network has >4 dimensions (4 can be graphed with some effort and a lot of color), and as such is literally beyond human comprehension via graphing. So what do we ...


3

I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that function IS. It's usually called the loss function (and, in general, objective function) and often denoted as $\mathcal{L}$ or $L$ (or something like that, i.e. it is not really important how you denote it). The specific function used as a ...


2

What you are describing is conceptually close to adversarial training. you should read more on adversarial examples and generative adversarial networks for more information. The idea is that there is a discriminator network, whose job is to correctly discriminate between positive and negative examples. We also have a generative network, that learns to ...


2

"The concept of a direction of fastest descent only makes sense in more than one dimension." https://math.stackexchange.com/a/180573


2

Don’t think about it as the being proportional to something. Think about it this way: I’m now at . Where do I want to be at Time step so that the error decreases? For that, I need to know how the error changes when I make small steps to the left or right of If increases as I increase (that is, if , then obviously, I would want to move a little bit to ...


2

Backpropagation with stride > 1 involves dilation of the gradient tensor with stride-1 zeroes. I created a blog post that describes this in greater detail.


2

One of the best ways to learn is use reference of others. Have a look at Peter Sadowski - Notes on Backpropagation (Page 3): There is also a great blog post by Eli Bendersky - The Softmax Function and Its Derivative.


2

Consider that you have a loss function, and you want to tune your model (network) to decrease the loss. The main concept is to tune parameters in a direction which decreases the loss and gives you a better model. You can imagine a mountain where you should reach to the lower grounds. There are 2 questions here. In which direction to move? How much should ...


2

The main point here is that you can write $Q(s, a|\theta) = R = \theta^\top \phi(s, a)$. For more details on this, you can read up on Policy Gradients (Chapter 13) from the 2nd edition of the Sutton and Barto book (in fact the expression you're looking for is equation 13.9) For simplicity, I'm setting $\beta=1$, but you can always put it in once you get the ...


2

So, the main doubt here is about the intuition behind the derivative part of back-propagation learning. First, I would like to point out 2 links about the intuition about how partial derivatives work Chain Rule Intuition and Intuitive reasoning behind the Chain Rule in multiple variables?. Now that we know how the chain rule works, lets see how we can use ...


2

The basic idea behind mini-batch training is rooted in the exploration / exploitation tradeoff in local search and optimization algorithms. You can view training of an ANN as a local search through the space of possible parameters. The most common search method is to move all the parameters in the direction that reduces error the most (gradient decent). ...


Only top voted, non community-wiki answers of a minimum length are eligible