4
$\begingroup$

Context and detail

I've been working on a particular image retrieval problem and I've found two popular threads in the literature:

Image retrieval (usually benchmarked with landmark retrieval datasets)

Face recognition/verification:

I'm still making my way through these lists and more (I've checked the ones I've looked at already) but I'm starting to get a sense that there's not much overlap in the techniques used, or the collective trains of thought in the research community. Here are the main points of divergence where I think both communities should be borrowing from each other.

  • Facial recognition seems to focus on getting embeddings to be as discriminative as possible by playing around with loss functions and training methods, whereas image retrieval seems to care more about ways of extracting feature descriptors from CNN pretrained backbones (types of pooling operations, which feature maps to look at, etc..).
  • Image retrieval has a considerable amount of work on what needs to happen after an embedding is obtained. Eg: dimensionality reduction, whitening + l2 norm, databise-side augmentation, query expansion, reranking etc
  • Facial recognition cares about keeping a minimum margin between non-matching faces in order to avoid mismatches, but I would think that should be imposed in image retrieval tasks as well (this is kind of a sub-point to my first point)

So to sum up: Why is it that facial recognition focuses on generating discriminative embeddings, while landmark retrieval focusses on generating rich "descriptors"? Why does landmark retrieval use this cool bag of tricks for database search while facial recognition just mentions kNN? Shouldn't all these considerations boost performance in either domain?

Improve this question
$\endgroup$
3
  • $\begingroup$ I have used opencv facial recogition and convolution neural network facial recognition. Are you interested in either? $\endgroup$
    – Golden Lion
    Commented Mar 8, 2021 at 12:43
  • $\begingroup$ Interested in deep learning approaches $\endgroup$
    – Alexander Soare
    Commented Mar 8, 2021 at 13:12
  • $\begingroup$ Keras cnn ? what is your input shape of the image $\endgroup$
    – Golden Lion
    Commented Mar 8, 2021 at 13:18

1 Answer 1

Reset to default
2
+50
$\begingroup$

Landmark retrieval has photographs of landmarks that you need to find out. Consider the degrees of freedom for this, the landmarks can many different colours (more than humans' faces) and also the colour range is all over the place (a landmark may be blue or white or red). The shapes of the various landmarks will also vary.

Now, consider face recognition problem. All humans faces look alike morphologically. If you look at the colour, it is not as varied as landmark recognition.

Because of the inherent data in both the problems, the research focuses on the diverging lines of thought. Rich descriptors are good for landmarks because the data itself is very rich and mired with variation. On the other hand, discriminative features are more desirable for face recognition because faces are more similar and less rich in variation, so differentiating between is hard.

It is the requirement of the problem that steers research in diverging directions.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks, that makes sense. May I ask, are you part of either of these research communities? Your answer is good, but I just want to know if I've heard it from the horse's mouth $\endgroup$
    – Alexander Soare
    Commented Mar 8, 2021 at 21:18
  • $\begingroup$ Yes, I do create and research facial recognition systems on a billion-scale. $\endgroup$
    – Abhishek Verma
    Commented Mar 9, 2021 at 8:38
  • $\begingroup$ Great, thank you. I'll leave this open for a bit and see if anyone else has something to say, then I'll close it once the bounty is rewarded. $\endgroup$
    – Alexander Soare
    Commented Mar 9, 2021 at 9:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .