8

I see no reason why decaying learning rates should create the kinds of jumps in losses that you are observing. It should "slow down" how quickly you "move", which in the case of a loss that otherwise consistently shrinks really should, at worst, just lead to a plateau in your losses (rather than those jumps). The first thing I observe in your code is that ...


7

Actually, the cross-entropy loss function would be appropriate here, since it measures the "distance" between a distribution $q$ and the "true" distribution $p$. You are right, though, that using a loss function called "cross_entropy" in many APIs would be a mistake. This is because these functions, as you said, assume a one-hot label. You would need to use ...


7

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write it. $$(\hat{y} - y)^2 = (\hat{y} - y)(\hat{y} - y) = \hat{y}^2 -2\hat{y}y + y^2$$ $$(y - \hat{y})^2 = (y -\hat{y})(y - \hat{y}) = y^2 -2y\hat{y} + \hat{y}^2$$ ...


6

In general a cost function can be negative. The more negative, the better of course, because you are measuring a cost the objective is to minimise it. A standard Mean Squared Error function cannot be negative. The lowest possible value is $0$, when there is no output error from any example input. How can our cost function which is mean squared error have ...


6

The Focus of This Question "How can ... we process the data from the true distribution and the data from the generative model in the same iteration? Analyzing the Foundational Publication In the referenced page, Understanding Generative Adversarial Networks (2017), doctoral candidate Daniel Sieta correctly references Generative Adversarial Networks, ...


6

The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is \begin{align} \frac{d}{d \hat{y}} \mathcal{L_1} &= \frac{d}{d \hat{y}} (\hat{y} - y)^2 \\ &= 2(\hat{y} - y) \frac{d}{d \hat{y}} (\hat{y} - y) \\ &= 2(\hat{y} - y) (1) \\ &...


5

The loss function used is the triplet loss function. Let me explain it part by part. Notation The $f^a_i$ means the anchor input image. The $f^p_i$ means the postive input image, which corresponds to the same people as the anchor image. The $f^n_i$ corresponds to the negative sample, which is a different person(input image) then the anchor image. The ...


3

In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations: what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima looking at the LF steepness we observe the Left LM is in a steeper region than ...


3

Let's start at the beginning. GANs are models that can learn to create data that is similar to the data that we give them. When training a generative model other than a GAN, the easiest loss function to come up with is probably the Mean Squared Error (MSE). Kindly allow me to give you an example (Trickot L 2017): Now suppose you want to generate cats ; ...


3

For the first question, RMSE and Euclidean distance have no difference, not that i know of. For the second question, you only need the common loss function for normal tasks. MSE is a common loss function used in linear regression tasks as well as loss function similar in nature like the RMSE. For classification tasks, Cross Entropy Loss is preferred. For ...


3

Depends on what does 1 represent in your task. If you are trying to predict household prices and 1 represents \$1, I think the average validation loss is good. If 1 represents \$10000 in this case, probably something is not right. But remember that there are 2 parts contributing to the overall loss. The mse loss and the l2 penalty loss. (Also remember that ...


3

The validation loss settles exactly at an error of one. Probably means there's something off with either the kind of data validation set has or with something in the training. An exact validation loss of one almost definitely means there's something off. I'd recommend before doing anything thoroughly go through your data or see if there's anything to debug ...


3

Welcome to AI Stack exchange! You're right, as the network is initialised randomly, the resultant function is essentially impossible to get your head around. This is because most of the time the network has >4 dimensions (4 can be graphed with some effort and a lot of color), and as such is literally beyond human comprehension via graphing. So what do we ...


3

I know that gradient descent allows you to find the local minimum of a function. What I don't know is what exactly that function IS. It's usually called the loss function (and, in general, objective function) and often denoted as $\mathcal{L}$ or $L$ (or something like that, i.e. it is not really important how you denote it). The specific function used as a ...


2

See the blog post Why You Should Use Cross-Entropy Error Instead Of Classification Error Or Mean Squared Error For Neural Network Classifier Training (2013) by James D. McCaffrey. It should give you an intuition of why the average cross-entropy (ACE) is more appropriate than MSE (but MSE is also applicable). In a few words, $\tanh$ + MSE is like sigmoid + ...


2

I don't think that the concept of generalization is (directly) related to the "shape" of the function close to the point where it attains a minimum. The concept of generalisation refers to when a trained model is able "perform well" on unseen data (that is, data not seen during the training phase). If a trained model does not generalise well, then it might ...


2

First of all you made a mistake, equation 8 in the paper is defined with $\frac{\partial L(\theta)}{\partial s_t}$ not $\frac{\partial L(\theta)}{\partial\theta}$. Loss is defined as: $L(\theta) = - \mathbb{E}_{w^s \sim p_{\theta}}[r(w^s)]$ If we use definition of expectation (for discrete case): $\mathbb{E}[X] = \sum\limits_{i} p_i(x_i)x_i$ we get ...


2

You can find an implementation of the REINFORCE algorithm (as defined in your question) in PyTorch at the following URL: https://github.com/JamesChuanggg/pytorch-REINFORCE/. First of all, I would like to note that a policy can be represented or implemented as a neural network, where the input is the state (you are currently in) and the output is a "...


2

But, if $t^i - o^i$ is negative, doesn't the power of 2 eliminate any negative result? In the loss function, yes that is correct, and is what you want - a measurement that gets higher due to any difference between the predicted and correct results. Minimising the value of that measurement is a goal for the optimiser. How can then exist any negative ...


2

The loss of the policy head here is really quite different from losses in, for instance, more "conventional" Supervised Learning approaches (where we typically expect/hope to see a relatively steady decrease in loss function). In this AlphaZero setup, the target that we're updating the policy head towards is itself changing during the training process. When ...


2

The derivative is the same as far as I understand it. If $y$ is constant and $\hat{y}$ is the variable the result will be: $((\hat{y} - y)^2)' = 2(\hat{y} - y)$ and for the other formula: $((y - \hat{y})^2)' = -2(y - \hat{y})$ which is the same.


2

Yes. Not only that, but error is highly noisy, prone to big spikes and sometimes quite long period of increase before decrease again or stabilize. Often it's even impossible to understand error plot without passing it through smoothing filter, so noisy it is. Specific depend on the problem of cause. It's not only for SGD but for any optimizer.


1

The true value $v_{\pi}(s)$ is a conceptual target for the $\overline{VE}$ in the book. You often do not know it in real problems. However, it is still used in two main ways in the book: Theoretically for analysis of different aprpoximation schemes, which can be shown to converge to minimise the $\overline{VE}$ objective, or a related one. In toy problems ...


1

Unfortunately no, the way to go is track the total reward and see if it's increasing and converging eventually. Value loss isn't a useful metric as the loss can be 0 when the value network always predicts 0 and the agent doesn't collect any reward, meaning very poor behavior.


1

Couple reccomendations: 1) I dont think your overfitting, your test loss is not ever increasing and is staying reasonbly proportional to train loss -- This may indicate that whatever loss your using is not a good indicator of the metric of interest (in this case, it seems you want that to be accuracy, but data is imbalnced so maybe look at avg precision?) 2) ...


1

Therefore,the network learns to always increase the Q function , and eventually the Q function is higher in same states in later learning steps If your value function keeps increasing in later steps that means that the network is still learning those Q-values, you shouldn't necessarily prevent that. Your Q-values won't increase forever even if the rewards ...


1

You can use discount factor gamma less then one. You can use finite time horizon - only for states which are no farther away then T time steps reward propagate back You can use sum of rewords averaged over time for Q All of those are legitimate approaches.


1

These answers are based on my personal understanding of Bert from both the paper and official_implementation, hope it will help: What do they mean by "maximum scoring span is used as the prediction"? As you know in SQuAD the input sequence is divided to 2 parts: Question and Document (from which we extract the answer if possible). Sometimes the input ...


1

This would mean that there is only one gradient update on your dataset of 16 samples. If you are taking the mean of all the squared errors ( mse ), then the loss of each sample is contributing to the 0.1 loss.


1

Mean squared error terms must be positive because the square of a number is positive. Therefore the sum (cost) is positive. The error is the difference between the hypothesis and the observation. I would focus on understanding why Ng seeks to minimize J and how it is minimization is achieved with partial derivatives via matrix implementation.


Only top voted, non community-wiki answers of a minimum length are eligible