Hot answers tagged

6

You should choose the model A. The loss is just a differentiable proxy for accuracy. That said, the situation should be examined in more detail. If the higher loss is due to the data term, examine the data which produce high loss and check for presence of overfitting or incorrect labels. If the higher loss is due to a regularizer then reducing the ...


5

I'll cover both L2 regularized loss, as well as Mean-Squared Error (MSE): MSE: L2 loss is continuously-differentiable across any domain, unlike L1 loss. This makes training more stable and allows for gradient-based optimization, as opposed to combinatorial optimization. Using L2 loss (without any regularization) corresponds to the Ordinary Least Squares ...


4

It's an uppercase "J" from the math calligraphy alphabet, i.e. \mathcal{J} in latex. $\mathcal{J}$


4

You should note that both your results are consistent with a "true" probability of 87% accuracy, and your measurement of a difference between these models is not statistically significant. With an 87% accuracy applied at random, then there is approx 14% chance of getting the two extremes of accuracy you have observed by chance if samples are chosen randomly ...


1

You can use the Exponential Moving Average method. This method is used in tensorbaord as a way to smoothen a loss curve plot. The algorithm is as follow: However there is a small problem doing it this way. As you can see S_t is initialized with the starting value, which makes the starting curve inaccurate. The green curve is the ideal curve for the ...


1

I found that the peak was caused by the data I am using. Specifically, the MinMaxScaler changed the data shape and I resolved the issue by simply dividing to the max value.


1

It depends on your application! Imagine a binary classifier that is always very "confident" - it always assigns P=100% to Class A and 0% to Class B, or vice versa (sometimes wrong, never uncertain!). Now imagine a "humble" model that is perhaps fractionally less accurate, but whose probabilities are actually meaningful (when it says "Class A with ...


1

do we also want to consider the subset of invalid actions for the $\max\limits_{a}Q(s_{t+1},a)$ No. Doing so would go against the theory behind the Bellman equation from which the update derives. The value of $r_{t+1} + \gamma \max\limits_{a'}Q(s_{t+1},a')$ needs to match to a realisable trajectory, otherwise the eventual expected values may be estimates ...


1

Assume we have a binary classification problem, which we want to solve with a simple single-layer perceptron. For a 2d space, a perceptron will have 2 inputs $x_1$ and $x_2$, and a bias denoted $x_0$ which is always $x_0=1$. It also has corresponding learnable weights $w_0$, $w_1$ and $w_2$. This can be vectorized: $$ \overline{x} = \begin{bmatrix} 1 \\ x_1 \...


1

The loss function is simply a way to measure how wrong a neural network is, it doesn't affect the output of the neuron. Say we have a neural network with 3 output neurons that attempts to classify images of cats, dogs, and humans. The output it gives is the confidence of the neural network's classification. For example if the output is [0, 0.2, 0.8] (0 being ...


1

Loss function is a function used to measure the loss. It is not used in any component of a neuron. It is used in updating the weights of the neuron i.e., in order to train the neuron. The contribution of a loss function is in the updation of $\bar{W}$. For a given $\bar{X}$ and $\bar{W}$, the neuron gives a post-action value $h$. But the desired output may ...


1

I assume you intended to write compute the evaluation metric over the validation set in batches; you do not compute loss over the validation set! That is quite a standard practice in many academic implementations (because, when the validation set is large enough, the memory will be a constraint), however, be sure to take the average of the values over all ...


1

The contradictory loss is the same loss function that the discriminator would normally use, except with deliberately incorrect labels. That is, when you train the generator, the output of the generator is fed to the discriminator, but instead of the correct label (typically $0$ for a false image), the opposite label is applied (e.g. $1$ for a real image). ...


1

Decrease of loss does not essentially lead to increase of accuracy (most of the time it happens but sometime it may not happen). To know why, you can have a look at this question. The network cares about decreasing the loss and it does not care about the accuracy at all. So it's no surprise to see what you presented. Additional note: If you use batch ...


Only top voted, non community-wiki answers of a minimum length are eligible