What loss function should I use if I only care about the accuracy of one class?

Question

CrossEntropyLoss optimizes the overall classification accuracy as $$ {n_{\text{correct}} \over N} $$

What loss function should I use if I only care about increasing the true positive rate of one class?

$$ {n_{\text{true A in predicted A}} \over N_{\text{predicted A}} } $$

For example, I predict 100 images to be in class A, and 90 out of this 100 are truly A. So the accuracy is 90%.

In the meantime, I predict another 900 images to be in class B, but 500 of them are actually A, and only 400 are B. So the overall accuracy is (90+400)/(100+900) = 49%.

In the meantime, I don't want $ N_{\text{predicted A}} $ to be too small, since one can see from above that a smaller $ N_{\text{predicted A}} $ can likely lead to larger true positive rate.

Answer 1

Cross entropy can also be weighted, standard practice when training on imbalanced datasets. Instead of the classic formulation

$$C = -\sum_{i=1}^{M}y_i\text{log}(\hat{y}_i)$$

you can rewrite it as

$$C = -\sum_{i=1}^{M}\sum_{c=1}^{N}w_cy_i\text{log}(\hat{y}_i)$$

were $w_c$ represent the weight of each training class. So if you care mostly about the accuracy of a specific class you could boost that specific class by using a large weight for it, let's say 0.8, and 0.2/(n-1) for all the remaining classes.