3
$\begingroup$

Context and detail

I've been working on a particular image retrieval problem and I've found two popular threads in the literature:

Image retrieval (usually benchmarked with landmark retrieval datasets)

Face recognition/verification:

I'm still making my way through these lists and more (I've checked the ones I've looked at already) but I'm starting to get a sense that there's not much overlap in the techniques used, or the collective trains of thought in the research community. Here are the main points of divergence where I think both communities should be borrowing from each other.

  • Facial recognition seems to focus on getting embeddings to be as discriminative as possible by playing around with loss functions and training methods, whereas image retrieval seems to care more about ways of extracting feature descriptors from CNN pretrained backbones (types of pooling operations, which feature maps to look at, etc..).
  • Image retrieval has a considerable amount of work on what needs to happen after an embedding is obtained. Eg: dimensionality reduction, whitening + l2 norm, databise-side augmentation, query expansion, reranking etc
  • Facial recognition cares about keeping a minimum margin between non-matching faces in order to avoid mismatches, but I would think that should be imposed in image retrieval tasks as well (this is kind of a sub-point to my first point)

So to sum up: Why is it that facial recognition focuses on generating discriminative embeddings, while landmark retrieval focusses on generating rich "descriptors"? Why does landmark retrieval use this cool bag of tricks for database search while facial recognition just mentions kNN? Shouldn't all these considerations boost performance in either domain?

$\endgroup$
3
  • $\begingroup$ I have used opencv facial recogition and convolution neural network facial recognition. Are you interested in either? $\endgroup$ – Golden Lion Mar 8 at 12:43
  • $\begingroup$ Interested in deep learning approaches $\endgroup$ – Alexander Soare Mar 8 at 13:12
  • $\begingroup$ Keras cnn ? what is your input shape of the image $\endgroup$ – Golden Lion Mar 8 at 13:18
1
+50
$\begingroup$

Landmark retrieval has photographs of landmarks that you need to find out. Consider the degrees of freedom for this, the landmarks can many different colours (more than humans' faces) and also the colour range is all over the place (a landmark may be blue or white or red). The shapes of the various landmarks will also vary.

Now, consider face recognition problem. All humans faces look alike morphologically. If you look at the colour, it is not as varied as landmark recognition.

Because of the inherent data in both the problems, the research focuses on the diverging lines of thought. Rich descriptors are good for landmarks because the data itself is very rich and mired with variation. On the other hand, discriminative features are more desirable for face recognition because faces are more similar and less rich in variation, so differentiating between is hard.

It is the requirement of the problem that steers research in diverging directions.

$\endgroup$
3
  • $\begingroup$ Thanks, that makes sense. May I ask, are you part of either of these research communities? Your answer is good, but I just want to know if I've heard it from the horse's mouth $\endgroup$ – Alexander Soare Mar 8 at 21:18
  • $\begingroup$ Yes, I do create and research facial recognition systems on a billion-scale. $\endgroup$ – Abhishek Verma Mar 9 at 8:38
  • $\begingroup$ Great, thank you. I'll leave this open for a bit and see if anyone else has something to say, then I'll close it once the bounty is rewarded. $\endgroup$ – Alexander Soare Mar 9 at 9:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.