1
$\begingroup$

I am having a video feed with multiple faces in it. I need to detect each face and the gender as well and assign the gender against each person. I am not sure how to uniquely identify a face as Face1 and Face2 etc. I do not need to know their names, just need to track a person in the entire video. I thought of tracking methods but people are constantly moving and changing so a box location can be occupied by another person after some frames.

I am interested in a way where I can assign an id to a face but I am not sure how to do it. I can use Facial Recognition Based embedding on each face and track that. But that seems to be an overkill for the job. Is there any other method available or Facial Recognition/Embedding is the only method to uniquely identify people in a video?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

Can I detect unique people in a video?

Yes. However, for some applications commercial systems are not good enough: German source

The concepts you are looking for are:

  • Optical Flow: for two frames, define for each point how it moved
  • Object tracking: given a single object in one frame, find it in the next. Optical flow helps with that.
  • Object detection: look at YOLO / SSD. Small introduction here.
Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks Martin, is object tracking robust if the object in interest are coming in and out of the frames. Optical flow will just track it if it is continuously in the stream. $\endgroup$
    – vc_dim
    Commented Nov 30, 2018 at 7:42
  • $\begingroup$ Correct. But Optical flow gives you a good idea where to look for new objects. $\endgroup$
    – Martin Thoma
    Commented Nov 30, 2018 at 7:54
  • $\begingroup$ Makes sense, Let me try that. Thanks a lot Martin. $\endgroup$
    – vc_dim
    Commented Nov 30, 2018 at 7:59
1
$\begingroup$

You can detect faces in video frames by employing the standard method of Viola-Jones detector (though there are sophisticated ones available now). You can get the paper from here. All the standard image processing libraries will have it implemented. You have to use the necessary API of that library.

This will give a bounding box around each of the detected face (it may miss out a few faces which are at different poses or illumination). You can now encode the detected face, inside the bounding box, into a vector using the features given by face2vec. In the face2vec model remove the classification part alone. This will give you a feature vector of size 905 dimension.

Now given two face detections, you can find the euclidean distance between their feature vectors. If it is below a threshold say $T$ then, you have detected same face, else you have detected two different faces.

Improve this answer
$\endgroup$
2
  • $\begingroup$ This is helpful Varsh, I will try face2vec. I am already trying encoding and graph clustering using euclidian distances. I get 128 dimension vector today. But I want to avoid Deep Learning Approaches as they are computation heavy and I want to deploy the system in a small device. Is there any other possible representation of the face which can help me cluster them. $\endgroup$
    – vc_dim
    Commented Nov 30, 2018 at 7:40
  • $\begingroup$ You can use PCA to perform dimensionality reduction on raw pixels and use the reduced size vector as features. In this case you have the control over the number of dimensions thereby reducing computations for your small device. $\endgroup$
    – varsh
    Commented Nov 30, 2018 at 8:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .