2
$\begingroup$

I've always found bounding box regression a bit weird. There's no positional encoding like in vision transformers, so how does the network "know" the absolute position when producing bounding box coordinates? It gets even weirder when we are dealing with two-stage detectors because, in the second stage bounding box regression, only an ROI is available, not the whole image.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

It of course depends on the detection model that is used.

But in your case I think you relate to a Faster-RCNN type architecture for bounding box detection. In this case only the relative position to an anchor is regressed, that is correct. Of course the regression "values" isolated have no information where they are absolute in the image since the same convolution is shared and slided over the whole image/layer. However, after your regression layer is done (i.e. slided the convolution over the whole previous layer) it outputs a new feature map with relative regression values. The regression values itself do not contain the information about the abolute position, but the xy position of the relative feature values inside the output feature map of the regressor combined with the relative values itself gives you the information about the absolute position.

This also makes it "translation invariant", which is an important and wanted feature of Faster-RCNN architectures.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yeah so we know the anchor positions from the start and we just need the relative scaling of the box. And if I recall correctly, each individual regression head is tied to a specific anchor right? $\endgroup$
    – Alexander Soare
    Commented Jan 12, 2022 at 18:43
  • 1
    $\begingroup$ Yes exactly. The "regression head"-convolution is shared and shifted over the whole input map and at each convolution-window position outputs 4k relative regression values that are written to the corresponding (absolute but unaccurate) position in the output map. Combined with the relative values themself you then get accurate absolute positions. $\endgroup$
    – Marcel_marcel1991
    Commented Jan 13, 2022 at 8:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .