In various neural network detection pipelines, the detection works as follows:
- One processes the input image through the pretrained backbone
- Some additional convolutional layers
- The detection head, where each pixel on the given feauture map predicts the following:
- Offset from the center of the cell ($\sigma(t_x), \sigma(t_y)$ on the image)
- Height and width of the bounding boxes $b_w, b_h$
- Objectness scores (probability of object presence)
- Class probabilities
Usually, detection heads produce not a single box, but multiple.
- The first version of YOLO - outputs 2 boxes per location on the feature map of size $7 \times 7$
- Faster R-CNN outputs 9 boxes per location
- YOLO v3 - outputs 9 boxes per pixel from the predefined anchors :
(10×13),(16×30),(33×23),(30×61),(62×45),(59× 119), (116 × 90), (156 × 198), (373 × 326)
These anchors give the priors for the bounding boxes, but with the help of $\sigma(t_x), \sigma(t_y), b_w, b_h$ one can get any possible bounding box on the image for some pixel on the feature map.
Therefore, the network will produce plenty of redundant boxes, and a certain procedure - NMS suppresion has to be run over the bounding box predictions to select only the best.
Or the purpose of these anchors is to start from a prior, reshape and shift slightly the bounding box, and then compare with the ground truth.
Is it the case, that if one used only a single bounding box for detection - it would be hard to train the network to rescale the initial bounding to, say, 10 times, and produce some specific aspect ratio?