The problem originated because of the nature of the code.
Code:
https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/classifier.py
model = SVC(kernel='linear', probability=True)
model.fit(emb_array, label)
class_names = [cls.name.replace('_', ' ') for cls in img_data]
As you see the code uses a SVC (Support Vector Classifier) to classify the classes. The SVC (or SVM) does not have an extra class for unknown
class.
For the threshold
variable, it is used in face detection, aka drawing a bounding box around the face for FaceNet to classify it.
Code:
https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/identify_face_image.py
frame = frame[:, :, 0:3]
bounding_boxes, _ = detect_face.detect_face(frame, minsize, pnet, rnet, onet, threshold, factor)
nrof_faces = bounding_boxes.shape[0]
As you can see, the threshold
variable is only used in detecting the bounding box.
Code for getting class name:
predictions = model.predict_proba(emb_array)
print(predictions)
best_class_indices = np.argmax(predictions, axis=1)
# print(best_class_indices)
best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices]
print(best_class_probabilities)
cv2.rectangle(frame, (bb[i][0], bb[i][1]), (bb[i][2], bb[i][3]), (0, 255, 0), 2) #boxing face
#plot result idx under box
text_x = bb[i][0]
text_y = bb[i][3] + 20
print('Result Indices: ', best_class_indices[0])
print(HumanNames)
You can see that no unknown
class can be found.
Solution
You can try adding another threshold value and check if the predictions
maximum value is lower than the threshold value. I have little experience in tensor flow so this is just a proof of concept, not sure if it will work.
best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices] #original code
if(best_class_probabilities < threshold_2):
best_class_indices = -1
HumanNames = "unknown"
By the way, because of the nature of triplet loss, you don't have to add and extra class to the SVC/SVM as the embedding model is locked and not trained, so unknown class embeddings will be very different to the known class. However you can try either approach.
Hope it can help you can have a nice day!