I am reading the R-CNN paper by Ross Girshick1 et al. (link) and I fail to understand how they do the inference. This is described in the section 2.2.Test-time Detection in the paper. I quote:
At test time, we run selective search on the test image to extract around 2000 region proposals (we use selective search’s “fast mode” in all experiments). We warp each proposal and forward propagate it through the CNN in order to read off features from the desired layer. Then, for each class, we score each extracted feature vector using the SVM trained for that class.
I do not understand how a Support Vector Machine (SVM) can score a feature vector since SVM does not tell you class probability, it only tells you if an object belongs to a class or not. How is this possible?
It seems that detection flow is: get image, run it through CNN and get feature vector, score this feature vector and run Non-Maximal Suppresion (NMS). But for running NMS we need the feature vector scored, and again, SVM do not score predictions, right?
Actually, when represented in the same paper, the SVM does not provide a score as you can see in the next image (taken from the same paper).
So, how this makes sense?