6 votes
Accepted

In which cases is the categorical cross-entropy better than the mean squared error?

As a rule of thumb, mean squared error (MSE) is more appropriate for regression problems, that is, problems where the output is a numerical value (i.e. a floating-point number or, in general, a real ...
user avatar
  • 34.4k
4 votes
Accepted

Why are there two versions of softmax cross entropy? Which one to use in what situation?

It's the same thing, first version is the special case of the more general one. In the first case you only have two classes, it's binary cross-entropy, and they also included iteration over batch of ...
user avatar
  • 2,236
3 votes

In which cases is the categorical cross-entropy better than the mean squared error?

In a classification problem it's better to get higher error and higher error slope when we predict the label wrong. As you see in the graph by using cross-entropy you get high error when the algorithm ...
user avatar
  • 131
2 votes
Accepted

How does the implementation of the VAE's objective function equate to ELBO?

I don't want to think about the correctness of your supposed ELBO equation now. Nevertheless, it's true that the ELBO can be rewritten in different ways (e.g. if you expand the KL divergence below, by ...
user avatar
  • 34.4k
2 votes

In which cases is the categorical cross-entropy better than the mean squared error?

We sometimes see that binary cross-entropy (BCE) loss is used for regression problems. This post is my opinion on using BCE for regression problems. The figure below is the plots of BCE, $-t*\log(x) -...
user avatar
2 votes
Accepted

Where can I find authentic references on "categorical cross entropy" and "categorical accuracy metric"?

Categorical just means that we will conduct multiclass classification. The output of the classifier is a binary vector. Each entry $x_i$ in the binary vector is a prediction whether or not the input ...
user avatar
  • 522
1 vote
Accepted

Is it appropriate to use a softmax activation with a categorical crossentropy loss?

Let's first recap the definition of the binary cross-entropy (BCE) and the categorical cross-entropy (CCE). Here's the BCE (equation 4.90 from this book) $$-\sum_{n=1}^{N}\left( t_{n} \ln y_{n}+\left(...
user avatar
  • 34.4k
1 vote
Accepted

Can the (sparse) categorical cross-entropy be greater than one?

Both the sparse categorical cross-entropy (SCE) and the categorical cross-entropy (CCE) can be greater than $1$. By the way, they are the same exact loss function: the only difference is really the ...
user avatar
  • 34.4k
1 vote
Accepted

How should I penalize the model proportionally to the error?

I want to make it so that if the correct label is 3, then it will penalize the model less heavily if it classifies a 4 than a 7 because 4 is closer numerically to 3 than 7 is. How do I do this? ...
user avatar
  • 24.5k

Only top scored, non community-wiki answers of a minimum length are eligible