7

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write it. $$(\hat{y} - y)^2 = (\hat{y} - y)(\hat{y} - y) = \hat{y}^2 -2\hat{y}y + y^2$$ $$(y - \hat{y})^2 = (y -\hat{y})(y - \hat{y}) = y^2 -2y\hat{y} + \hat{y}^2$$ ...


6

There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include: Probabalistically accepting worse solutions in the hope that this will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing). Maintaining a list of recently-encountered states (or attributes ...


6

The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is \begin{align} \frac{d}{d \hat{y}} \mathcal{L_1} &= \frac{d}{d \hat{y}} (\hat{y} - y)^2 \\ &= 2(\hat{y} - y) \frac{d}{d \hat{y}} (\hat{y} - y) \\ &= 2(\hat{y} - y) (1) \\ &...


5

In general $|f(x) - f_k(x)| \leq \epsilon$ doesn't ensure $|\nabla f(x) - \nabla f_k(x)| \leq c\epsilon$. And for neural networks there is no reason to believe it will happen either. You can also look at Sobolev Training Paper (https://arxiv.org/abs/1706.04859). In particular, note that Sobolev training was better than critic training, which indirectly may ...


5

In general a cost function can be negative. The more negative, the better of course, because you are measuring a cost the objective is to minimise it. A standard Mean Squared Error function cannot be negative. The lowest possible value is $0$, when there is no output error from any example input. How can our cost function which is mean squared error have ...


5

$g(x) = x^2$ is indeed a parabola and thus has just one optimum. However, the $\text{MSE}(\boldsymbol{x}, \boldsymbol{y}) = \sum_i (y_i - f(x_i))^2$, where $\boldsymbol{x}$ are the inputs, $\boldsymbol{y}$ the corresponding labels and the function $f$ is the model (e.g. a neural network), is not necessarily a parabola. In general, it is only a parabola if $...


4

I think Musk was using the terminology correctly though perhaps with hyperbole. I believe this was tweeted in the context of the botnet attacks on name-resolution services that broke Netflix and a large number of other internet services for a time. He was expressing the idea that you could train a botnet-based system to attack the internet by giving it a ...


4

Calculation of gradient \begin{align} \nabla_{\theta} \log(\pi_{\theta}(a|s)) &= \phi(s,a) - \mathbb E[\phi (s, \cdot)]\\ &= \phi(s,a) - \sum_{a'} \pi(a'|s) \phi(s,a') \end{align} is only valid for linear function approximation with action preferences of form \begin{equation} h(s, a, \theta) = \theta^T \phi(s,a) \end{equation} and softmax policy \...


3

It is not possible to backpropagate gradients through a layer with non-differentiable functions. However, the pooling layer function is differentiable*, and usually trivially so. For example: If an average pooling layer has inputs $z$ and outputs $a$, and each output is average of 4 inputs then $\frac{da}{dz} = 0.25$ (if pooling layers overlap it gets a ...


3

If gradients are noisy (does this means that in some dimension we have small and in some high curvature or that error noise differs for very similar values of w?) Gradients being noisy means that they are "inconsistent" across different epochs / training steps. With that I mean that they'll sometimes point in one direction, later in a different (maybe the ...


3

This is standard backpropagation. The gradient term you see is in fact a vector of partial derivatives where each element is the partial derivative of the log-likelihood with respect to each element of the parameter vector \theta. Therefore, it has the same dimensionality as \theta. Each element of the parameter vector is then updated with the respective ...


3

There are 3 separate issues that are often confounded in Deep Learning and Neuroscience: Deep Learning is inspired by the way the biological brain works. Deep Learning is how the biological brain works. Deep Learning can model how the biological brain works. Number 1 is accurate. The brain has many layers and many connections. Those principles have ...


3

There are several different algorithms that can be used for gradient free neural network training. Some of these algorithms include particle swarm optimization, genetic algorithms, simulated annealing, and several others. Almost any optimization algorithm can be used to train a neural network. Here is an overview of some of the algorithms I listed: Particle ...


3

There is not single answer to the vanishing gradient problem. However, there a few things that can help. As mentioned in the comments, use of Rectified Linear Units (ReLU) as your activation function can help, since the it does not get saturated for large neuron inputs. Next, careful choice of weight initialization can help avoid saturation, as well. See ...


3

DQN "library" implementations that I have seen do use mini-batches to train, and I would generally recommend this, as it usually strikes a reasonable balance between number of weight updates and accuracy of the gradients. In your first link, and the code excerpt, the sample list is literally called minibatch. However, the developer then goes on to make a ...


3

In a setup like the above where the derivat[iv]e is 0 is it true that an NN won´t learn anything? There are a couple of adjustments to gradients that might apply if you do this in a standard framework: Momentum may cause weights to continue changing if any recent ones were non-zero. This is typically implemented as a rolling mean of recent gradients. ...


3

In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations: what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima looking at the LF steepness we observe the Left LM is in a steeper region than ...


3

NEAT is a genetic algorithm (GA). A genetic algorithm maintains a population of individuals (or chromosomes) and evolves it using operations like the crossover or the mutation, so that the fittest individuals keep living and most other individuals die. The nature of the individuals depends on the problem. For example, in the case of NEAT, the individuals are ...


2

What you are describing is conceptually close to adversarial training. you should read more on adversarial examples and generative adversarial networks for more information. The idea is that there is a discriminator network, whose job is to correctly discriminate between positive and negative examples. We also have a generative network, that learns to ...


2

"The concept of a direction of fastest descent only makes sense in more than one dimension." https://math.stackexchange.com/a/180573


2

Don’t think about it as the being proportional to something. Think about it this way: I’m now at . Where do I want to be at Time step so that the error decreases? For that, I need to know how the error changes when I make small steps to the left or right of If increases as I increase (that is, if , then obviously, I would want to move a little bit to ...


2

Backpropagation with stride > 1 involves dilation of the gradient tensor with stride-1 zeroes. I created a blog post that describes this in greater detail.


2

One of the best ways to learn is use reference of others. Have a look at Peter Sadowski - Notes on Backpropagation (Page 3): There is also a great blog post by Eli Bendersky - The Softmax Function and Its Derivative.


2

Consider that you have a loss function, and you want to tune your model (network) to decrease the loss. The main concept is to tune parameters in a direction which decreases the loss and gives you a better model. You can imagine a mountain where you should reach to the lower grounds. There are 2 questions here. In which direction to move? How much should ...


2

The main point here is that you can write $Q(s, a|\theta) = R = \theta^\top \phi(s, a)$. For more details on this, you can read up on Policy Gradients (Chapter 13) from the 2nd edition of the Sutton and Barto book (in fact the expression you're looking for is equation 13.9) For simplicity, I'm setting $\beta=1$, but you can always put it in once you get the ...


2

So, the main doubt here is about the intuition behind the derivative part of back-propagation learning. First, I would like to point out 2 links about the intuition about how partial derivatives work Chain Rule Intuition and Intuitive reasoning behind the Chain Rule in multiple variables?. Now that we know how the chain rule works, lets see how we can use ...


2

The basic idea behind mini-batch training is rooted in the exploration / exploitation tradeoff in local search and optimization algorithms. You can view training of an ANN as a local search through the space of possible parameters. The most common search method is to move all the parameters in the direction that reduces error the most (gradient decent). ...


2

Here are a few guidelines, inspired by the deep learning specialization course, to choose the size of the mini-batch: If you have a small training set, use batch gradient descent (m < 200) In practice: Batch mode: long iteration times Mini-batch mode: faster learning Stochastic mode: lose speed up from vectorization The typically mini-batch sizes ...


2

From this awesome blog How to Configure Mini-Batch Gradient Descent Mini-batch gradient descent is the recommended variant of gradient descent for most applications, especially in deep learning. Mini-batch sizes, commonly called “batch sizes” for brevity, are often tuned to an aspect of the computational architecture on which the implementation is being ...


2

One possible cause of SARSA with a linear approximator failing to learn a good policy for the Mountain Car problem is that you are not encoding the environment state, the position and velocity. In David Silver's Lecture 6 that you mention in your question, he describes the state encoding, which in this case is a tile coding, here: https://youtu.be/...


Only top voted, non community-wiki answers of a minimum length are eligible