I've been reading the Fast R-CNN paper.
My understanding is that the input to one forward pass is the whole input image plus a list of RoIs (generated by selective search or another region proposal method). Then I understand that on the last convolution layer's feature map (let's call it FM), each corresponding RoI gets RoI-pooled, where now the corresponding ROIs are a rectangular (over height and width) slice of the FM tensor over all channels.
But I'm having trouble with two concepts:
How is the input RoI mapped to the corresponding RoI in FM? Each neuron comes from a very wide perceptive field, so, in a deep neural network, there's no way of making a 1:1 mapping between input neurons and neurons in the last convolution layer right?
Disregarding that I'm confused in point 1, once we have a bunch of RoIs in FM and we do the RoI pooling, we have N pooled feature vectors. Do we now run each of these through one FC network one by one? Or do we have N branches of FC networks? (that wouldn't make sense to me)
I have also read the faster R-CNN paper. In the same way, I'm also interested to know about how the proposed regions from RPN map to the input of the RoI pooling in the Fast R-CNN layers. Because actually those proposed regions live in the space of the input image, not in the space of the deep feature map.