How to incorporate a symmetry constraint in the loss function to train a CNN?

Question

I have a task of extremely sparse binary segmentation, i.e. the segmentation mask contains either 0 or 1, and there are ~95% zeros and only ~5% ones. I use the focal loss to address the sparseness (which is equivalent in my case to imbalances). I have another piece of information that I want to incorporate in the loss term.

The desired output is always symmetric over the diagonal. I was searching for a way to use this information in the loss, but I couldn't find a solution. How would I do this?

For some example of the symmetry in the segmentation maps, I added an arrow to show the axis of symmetry:

Answer 1

If you know it is symmetric, then you could do a couple things.

1. Zero out a half.

Don't bother learning both halves of the image. Just put a zero mask over the upper or lower half of the output matrix and just have the network regress the other half. Just don't make the network do more work than it needs to do.

2. Learn both, but add symmetric loss

In your case, it looks like you could create two loss functions added together.

$focal(x, y) + focal(x^T, y)$

This will help the network learn both halves equally.

3. L1 between $x$ and $x^T$

This might be silly but adding a Huber loss between $x$ and $x^T$ might help promote symmetry, but I'm not as much if a fan of it. Personally I'm more partial to (1).

4. Combine (1) and (2)

You could take the loss function of (2) at train time but at test time just use whatever half had better metrics and copy that to the bottom half.

5. Post Processing

Add a post processing custom layer that takes the average of the two halves so that you guarantee symmetry.

$x' = \frac{1}{2}(x + x^T)$

Then do your normal focal loss. Personally I like this the best since it always guarantees a symmetric output and is a pretty easy custom layer.