7
votes
Accepted
In which cases is the categorical cross-entropy better than the mean squared error?
As a rule of thumb, mean squared error (MSE) is more appropriate for regression problems, that is, problems where the output is a numerical value (i.e. a floating-point number or, in general, a real ...
- 37.1k
5
votes
Accepted
Why are there two versions of softmax cross entropy? Which one to use in what situation?
It's the same thing, first version is the special case of the more general one. In the first case you only have two classes, it's binary cross-entropy, and they also included iteration over batch of ...
- 2,286
3
votes
In which cases is the categorical cross-entropy better than the mean squared error?
We sometimes see that binary cross-entropy (BCE) loss is used for regression problems. This post is my opinion on using BCE for regression problems.
The figure below is the plots of BCE, $-t*\log(x) -...
- 131
3
votes
In which cases is the categorical cross-entropy better than the mean squared error?
In a classification problem it's better to get higher error and higher error slope when we predict the label wrong.
As you see in the graph by using cross-entropy you get high error when the algorithm ...
- 131
3
votes
Accepted
How does the implementation of the VAE's objective function equate to ELBO?
I don't want to think about the correctness of your supposed ELBO equation now. Nevertheless, it's true that the ELBO can be rewritten in different ways (e.g. if you expand the KL divergence below, by ...
- 37.1k
2
votes
Accepted
Can the (sparse) categorical cross-entropy be greater than one?
Both the sparse categorical cross-entropy (SCE) and the categorical cross-entropy (CCE) can be greater than $1$. By the way, they are the same exact loss function: the only difference is really the ...
- 37.1k
2
votes
Accepted
Where can I find authentic references on "categorical cross entropy" and "categorical accuracy metric"?
Categorical just means that we will conduct multiclass classification. The output of the classifier is a binary vector. Each entry $x_i$ in the binary vector is a prediction whether or not the input ...
- 532
1
vote
Accepted
Is it appropriate to use a softmax activation with a categorical crossentropy loss?
Let's first recap the definition of the binary cross-entropy (BCE) and the categorical cross-entropy (CCE).
Here's the BCE (equation 4.90 from this book)
$$-\sum_{n=1}^{N}\left( t_{n} \ln y_{n}+\left(...
- 37.1k
1
vote
Accepted
How should I penalize the model proportionally to the error?
I want to make it so that if the correct label is 3, then it will penalize the model less heavily if it classifies a 4 than a 7 because 4 is closer numerically to 3 than 7 is. How do I do this?
...
- 26.6k
Only top scored, non community-wiki answers of a minimum length are eligible