6

As a rule of thumb, mean squared error (MSE) is more appropriate for regression problems, that is, problems where the output is a numerical value (i.e. a floating-point number or, in general, a real number). However, in principle, you can use the MSE for classification problems too (even though that may not be a good idea). MSE can be preceded by the ...


4

It's the same thing, first version is the special case of the more general one. In the first case you only have two classes, it's binary cross-entropy, and they also included iteration over batch of samples. In the second case you have multiple classes and in the current form it's only for a single sample. In the first case there is only one output, if you ...


3

In a classification problem it's better to get higher error and higher error slope when we predict the label wrong. As you see in the graph by using cross-entropy you get high error when the algorithm predict a label wrong and small error when the prediacted label is close enough, so it helps us to separate the predicted classes better.


2

We sometimes see that binary cross-entropy (BCE) loss is used for regression problems. This post is my opinion on using BCE for regression problems. The figure below is the plots of BCE, $-t*\log(x) - (1-t)*\log(1-x)$, for several target values $t = 0.0, 0.1, ..., 0.5$. (The plots for $t>0.5$ are mirror images of those for $t<0.5$, so I omitted them.) ...


1

Let's first recap the definition of the binary cross-entropy (BCE) and the categorical cross-entropy (CCE). Here's the BCE (equation 4.90 from this book) $$-\sum_{n=1}^{N}\left( t_{n} \ln y_{n}+\left(1-t_{n}\right) \ln \left(1-y_{n}\right)\right) \label{1}\tag{1},$$ where $t_{n} \in\{0,1\}$ is the target $y_n \in [0, 1]$ is the prediction (as produced by ...


1

Both the sparse categorical cross-entropy (SCE) and the categorical cross-entropy (CCE) can be greater than $1$. By the way, they are the same exact loss function: the only difference is really the implementation, where the SCE assumes that the labels (or classes) are given as integers, while the CCE assumes that the labels are given as one-hot vectors. Here ...


1

I don't want to think about the correctness of your supposed ELBO equation now. Nevertheless, it's true that the ELBO can be rewritten in different ways (e.g. if you expand the KL divergence below, by applying its definition, you will end up with a different but equivalent version of the ELBO). I will use the most common (and definitely most intuitive, at ...


1

I want to make it so that if the correct label is 3, then it will penalize the model less heavily if it classifies a 4 than a 7 because 4 is closer numerically to 3 than 7 is. How do I do this? Really you should not, because the symbols used (Arabic numerals) do not have direct relation to quantity in the same way e.g. tally counts or dots do. They are good ...


Only top voted, non community-wiki answers of a minimum length are eligible