I am training a NN model. The data is highly imbalanced (3% for positive labels), and I have not resampled more true classes in the training set. The model performs much better when categorical cross-entropy is used.

My hypothesis is categorical cross-entropy requires 2 logits rather than 1. So the network does not have to suppress the logit for the true class that much when the model sees samples with false classes.

Does my hypothesis makes sense, and is there any conclusion on this topic? E.g., which one is better for a highly imbalanced binary classification dataset.

  • $\begingroup$ What do you specifically mean by "The model performs much better"? How are you assessing the performance of the model? Which metric are you computing and on which dataset? $\endgroup$
    – nbro
    Dec 29 '21 at 10:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.