Dice loss gives binary output whereas binary crossentropy produces probability output map

Question

On recommendation of Kanak on stackoverflow I am posting this question here:

Currently I am experimenting with various loss functions and optimizers for my binary image segmentation problem. The loss functions that I use in my Unet however give different output segmentation maps.

I have a highly imbalanced dataset, thus I am trying dice loss for which the customized function is given below.

    def dice_coef(y_true, y_pred, smooth=1):
        """
        Dice = (2*|X & Y|)/ (|X|+ |Y|)
             =  2*sum(|A*B|)/(sum(A^2)+sum(B^2))
        ref: https://arxiv.org/pdf/1606.04797v1.pdf
        """
        intersection = K.sum(K.abs(y_true * y_pred), axis=-1)
        return (2. * intersection + smooth) / (K.sum(K.square(y_true), -1) + K.sum(K.square(y_pred), -1) + smooth)

    def dice_coef_loss(y_true, y_pred):
        return 1 - dice_coef(y_true, y_pred)

Binary cross entropy results in a probability output map, where each pixel has a color intensity that represents the chance of that pixel being the positive or negative class. However, when I use the dice loss function, the output is not a probability map but the pixels are classed as either 0 or 1.

My questions are:

1.How is it possible that these different loss functions have these vastly different results?

2. Is there a way to customize the dice loss function so that the output segmentation map is a probability map similar to the one of binary crossentropy loss.

Answer 1

The probability map / output isn't produced by your loss function, but your output layer, which is activated either by softmax or sigmoid.

In other words, your dice loss output is also a probability map. It's simply very confident in itself. If you forget about the problem with potential overfitting for a moment and train your binary crossentropy model longer, the probability values will eventually all converge to the 2 ends (0 and 1).

In my experience, dice loss and IOU tend to converge much faster than binary crossentropy for semantic segmentation, so if you stop the training early on, dice loss will produce a probability map that resembles a binarized output more so than binary crossentropy.