According to the Mask R-CNN paper and the picture below (taken from the paper), the mask branch is computed in parallel with the bbox classification and regression branches.
However in the paper they write that inference is done differently from training, not in parallel:
Inference: At test time, the proposal number is 300 for the C4 backbone (as in [36]) and 1000 for FPN (as in [27]). We run the box prediction branch on these proposals, followed by non-maximum suppression [14]. The mask branch is then applied to the highest scoring 100 detection boxes. Although this differs from the parallel computation used in training, it speeds up inference and improves accuracy (due to the use of fewer, more accurate RoIs).
How is this actually done? By construction, the masks are outputted in parallel with the bounding boxes. So how can they run the mask branch after the bounding box prediction? Do they run it twice?
