The paper referenced by Martin Thoma is the go-to for semantic segmentation. However I will also like to add the Panoptic Segmentation metric as an aggregated method to measure both the detection task and segmentation task of the model.
It is a very well-known and widely used metric since it is the standard metric for COCO dataset (segmentation)
This is the ...
You can create a mapping from classes to colors for a simple one is:
y # y.shape = (W, H, n_classes)
_, y_color = y.max(dim=-1, keepdim=True) / n_classes # y_color.shape = (W, H, 1)
y_color = torch.cat([y_color] * 3, dim=-1) # y_color.shape = (W, H, 3)
(using pytorch like code)
This mapping is visualizable, of course, you may get nicer visualizations if ...