# Tag Info

5

You should choose the model A. The loss is just a differentiable proxy for accuracy. That said, the situation should be examined in more detail. If the higher loss is due to the data term, examine the data which produce high loss and check for presence of overfitting or incorrect labels. If the higher loss is due to a regularizer then reducing the ...

4

I'll cover both L2 regularized loss, as well as Mean-Squared Error (MSE): MSE: L2 loss is continuously-differentiable across any domain, unlike L1 loss. This makes training more stable and allows for gradient-based optimization, as opposed to combinatorial optimization. Using L2 loss (without any regularization) corresponds to the Ordinary Least Squares ...

3

You should note that both your results are consistent with a "true" probability of 87% accuracy, and your measurement of a difference between these models is not statistically significant. With an 87% accuracy applied at random, then there is approx 14% chance of getting the two extremes of accuracy you have observed by chance if samples are chosen randomly ...

1

I assume you intended to write compute the evaluation metric over the validation set in batches; you do not compute loss over the validation set! That is quite a standard practice in many academic implementations (because, when the validation set is large enough, the memory will be a constraint), however, be sure to take the average of the values over all ...

1

The contradictory loss is the same loss function that the discriminator would normally use, except with deliberately incorrect labels. That is, when you train the generator, the output of the generator is fed to the discriminator, but instead of the correct label (typically $0$ for a false image), the opposite label is applied (e.g. $1$ for a real image). ...

1

Decrease of loss does not essentially lead to increase of accuracy (most of the time it happens but sometime it may not happen). To know why, you can have a look at this question. The network cares about decreasing the loss and it does not care about the accuracy at all. So it's no surprise to see what you presented. Additional note: If you use batch ...

1

You can use the Exponential Moving Average method. This method is used in tensorbaord as a way to smoothen a loss curve plot. The algorithm is as follow: However there is a small problem doing it this way. As you can see S_t is initialized with the starting value, which makes the starting curve inaccurate. The green curve is the ideal curve for the ...

1

I found that the peak was caused by the data I am using. Specifically, the MinMaxScaler changed the data shape and I resolved the issue by simply dividing to the max value.

1

It depends on your application! Imagine a binary classifier that is always very "confident" - it always assigns P=100% to Class A and 0% to Class B, or vice versa (sometimes wrong, never uncertain!). Now imagine a "humble" model that is perhaps fractionally less accurate, but whose probabilities are actually meaningful (when it says "Class A with ...

Only top voted, non community-wiki answers of a minimum length are eligible