# Tag Info

4

I'll cover both L2 regularized loss, as well as Mean-Squared Error (MSE): MSE: L2 loss is continuously-differentiable across any domain, unlike L1 loss. This makes training more stable and allows for gradient-based optimization, as opposed to combinatorial optimization. Using L2 loss (without any regularization) corresponds to the Ordinary Least Squares ...

4

It seems your question is concerned with how an empirical mean works. It is indeed true that, if all $x^{(i)}$ are independent identically distributed realisations of a random variable $X$, then $\lim_{n \rightarrow \infty} \frac{1}{n}\sum_{i=1}^n f(x^{(i)}) = \mathbb{E}[f(X)]$. This is a standard result in statistics known as the law of large numbers.

3

When you use the softmax activation function is usually as a last layer of your network and to get an output that is a vector. Now your confusion is about shapes, so let's review a bit of calculus. If you have a function $$f:\mathbb{R}\rightarrow\mathbb{R}$$ the derivative is a function on its own and you have $$f':\mathbb{R}\rightarrow\mathbb{R}.$$ If you ...

3

I will first explain briefly to you the difference between supervised learning and reinforcement learning to make sure that you don't have any misunderstandings. In supervised learning you are provided with some data $\{(\textbf{x}_i, y_i)\}_{i=1}^n$ where $\textbf{x}_i$ are the features for data point $i$ and $y_i$ is its true label. Now, the aim of ...

3

I took a look at the Tensor2Tensor's source code implementation as per @nbro's suggestion, and it seems like the loss function is the cross entropy between the predicted ($\|sentence length\|$ x $\|vocab\|$) probability matrix (right before taking the argmax to find the token to output), and the $\|sentence length\|$-length vector of token IDs as the true ...

3

I've implemented this exact scenario before; your approach would most likely be successful, but I think it could be simplified. Therefore, when deciding on which action to pick, agent sets Q-values to 0 for all the illegal moves while normalizing the values of the rest. In DQN, the Q-values are used to find the best action. To determine the best action in ...

2

We sometimes see that binary cross-entropy (BCE) loss is used for regression problems. This post is my opinion on using BCE for regression problems. The figure below is the plots of BCE, $-t*\log(x) - (1-t)*\log(1-x)$, for several target values $t = 0.0, 0.1, ..., 0.5$. (The plots for $t>0.5$ are mirror images of those for $t<0.5$, so I omitted them.) ...

2

First let us note the definition of the advantage function: $$A(s,a) = Q(s,a) - V(s) \; ,$$ where $Q(s,a)$ is the action-value function and $V(s)$ is the state-value function. In theory you could represent these by two different function approximators, but this would be quite inefficient. However, note that $$Q(s,a) = \sum_{s',r} \mathbb{P}(s',r|s,a)(r + ... 2 The idea in PPO is that you want to reuse the batch many times to update the current policy. However, you cannot update mindlessly in a regular actor-critic fashion, because your policy might stray too far away from the optimal point. This means you repeat your step 6. epoch amount of times for the same batch of trajectories. Usually epoch is somewhere ... 2 In deep learning, the error function is sometimes known as loss function or cost function (but I do not exclude the possibility that these terms/expressions have also been used to refer to different although related functions, so you should take into account your context). In statistical learning theory, the function that you want to minimize is known as ... 2 I see why you might be confused. First, the logistic-loss or log-loss is technically called cross-entropy loss. This function is very simple: CE = -[y \log(p) + (1 - y) \log(1 - p)] This tells basically if the predicted class y was right y=1 then the loss is CE=-\log(p), if the predicted class was not the right one then the loss is CE=-\log(1-p). ... 2 It is important to note that the exact statement is the eqation given below can never be 0 for misclassified points in  S^+$$ E(X) = (y - \text{sign}\{\overline{W} \cdot \overline{X}\}) $$And S+ is defined as the set of all misclassified training points X \in S that satisfy the condition y(\overline{W} \cdot \overline{X})<0  which means that y ... 2 On page 5 of the VAE paper, it's clearly stated We let p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z}) be a multivariate Gaussian (in case of real-valued data) or Bernoulli (in case of binary data) whose distribution parameters are computed from \mathbf{z} with a MLP (a fully-connected neural network with a single hidden layer, see appendix \mathrm{... 2 Yes, you can do that, and it is a standard practice. One famous example is the "Inception" network architecture. To keep inner subnets from "dying out", several outputs from inner layers are extracted and passed into FC->Softmax. Then all the outputs are averaged in the loss function. From practical point of view, you won't be able to ... 2 The Bellman equation in RL is usually defined$$v_\pi(s) = \sum_a \pi(a|s) \sum_{s', r} p(s', r|s, a)\left[r + v_\pi(s')\right] = \mathbb{E}_{s' \sim p, a \sim \pi}\left[r(s, a) + v_\pi(s')\right] \; .$$The way you have written it is correct, but I just thought I would point this out. Regardless, your intuition is correct in that it expresses a recursive ... 2 If you know it is symmetric, then you could do a couple things. Zero out a half. Don't bother learning both halves of the image. Just put a zero mask over the upper or lower half of the output matrix and just have the network regress the other half. Just don't make the network do more work than it needs to do. Learn both, but add symmetric loss In your ... 1 In many cases, a fitness function (FF) is indeed similar to a reward function (RF), but, in other cases, it's more similar to a cost function (CF) as used in supervised learning (SL), and I explain below why. The FF, RF, and CF are used to evaluate the individuals, actions, and predictions, respectively, hence they can all be thought of as evaluation ... 1 After further research, I have found the answer. nbro was of course right, the weighting is not implementation dependant, it was already introduced in the paper (arXiv). However, there is minimal information about that, only it is mentioned within the optimization function: \arg \min_G \max_D \mathcal{L}_{cGAN}(G,D) + \lambda \mathcal{L}_{L1}(G) In fact, ... 1 There is no sign error and we should not change to \arg\max. With Policy Gradients I find that it is not useful to think about things such as a 'loss'. In short, we want to first find the derivative of the RL objective J(\theta) = v_\pi(s_0), where \pi is our policy that depends on some parameters \theta. The policy gradient theorem tells us that$$\...

1

They are not maximizing the gradient, the gradient is of the form $$\nabla_{\theta} J \approx \sum_{t=0}^T G_t \nabla_{\theta} \log(\pi_{\theta}(a_t|s_t))$$ that means that when implementing it in software you can form your objective as $$J = \sum_{t=0}^T G_t \log(\pi_{\theta}(a_t|s_t))$$ and then ...

1

Since you're using categorical cross-entropy loss, the last layer (output layer) should come with softmax activation instead of identity (as being blank in layers.Dense(4)). model = tf.keras.Sequential([ normalize, ..., layers.Dense(4, activation=tf.keras.activations.softmax) ]) And SparseCategoricalCrossentropy is different from ...

1

To provide a good answer would fill several pages. To keep it very simple try many different loss functions on your model. Your goal is to have the highest performance based on some desired prediction metric (e.g., RMSE, MAE, MAPE, etc.). You almost always have plenty of time to try many loss functions so you don't need to have a full understanding, and ...

1

A critical goal of training a neural network is to minimize the loss. Loss is not explained for spaCy because it is a general concept for machine learning and deep learning. Loss is not specific to spaCy and although there are some finer details I don't believe that is your inquiry. In general, to understand loss functions, I recommend the following ...

1

Yes, $E$ is the cross-entropy function and a direct generalization of the binary case. For the binary case, probability to belong to the class $1$ is given by a sigmoid function $\sigma(x)$ of the output $x$, and the probability to belong to the class $0$ is $1 - \sigma(x)$. Therefore the binary crossentropy will give:  -\sum_i (l_i \log \sigma(x) + (1 - ...

1

It depends on the complexity of your problem. $\mathbb{R}^2 \rightarrow \mathbb{R}^1$ looks simple, but I can give you some nonsense complicated examples that need a deep network. So, the complexity of the problem sets the number of layers and neurons. The kind of problem will determine the architecture of your network (if it needs memory or not). In most ...

1

Can my loss function be evaluating the model until it dies? 1/survival time could be the loss value to be minimized by gradient descent. In order to use backpropagation and gradient descent, you have to relate the loss function directly to the output of the neural network. Your proposed loss function is too indirect, it is not possible to turn it directly ...

1

Just realised the issue - super subtle (at least for a Python novice like me) - I implemented numerical gradient checking (as I should have done from the start) and saw the gradient descent was working incorrectly. The dataset Y was created as a datatype "uint8". In the line of the back-propagation where I calculate the derivative with respect to &...

1

The loss function (aka cost function) measures the correctness of the predictions of the model. For example, a simple cost function could be $|y - f(x)|$, where $\hat{y} = f(x)$ is the prediction of the model $f$ when the input is $x$, $y$ is the ground-truth label for input $x$ (i.e. what the model is supposed to output when the input is $x$), and \$|\...

1

I don't think there's a way of doing what you want, at least, I've never seen such a thing (and, currently, I am not seeing how this could be done in the general case). The same neural network model but with different (or same) weights could have been trained with the same loss function or not. For example, although it may not be a good idea, you can train ...

1

The advantage is basically a function of the actual return received and a baseline. The function of the baseline is to make sure that only the actions that are better than average receive a positive nudge. One way to estimate the baseline is to have a value function approximator. At every step, you train a NN, using the trajectories collected via the ...

Only top voted, non community-wiki answers of a minimum length are eligible