1
$\begingroup$

My project requires gender recognition of people shown on the given images, with more than one person per image. However, these people can be positioned in frontal or side view(passing by perpendicularly, no face visible). On the pictures there will be entire bodies shown, not only the faces. My idea is to firstly use object detection to point where people would be and next use CNNs to recognize gender of each person.

My question is: should I use one object detection algorithm for both frontal and side views of a person and then classify them with one CNN, or should I use object detection to separately find people positioned in frontal and side manner and use two different CNNs, one for classification of frontal views and one for side views?

I am asking this because I think it might be easier for one NN to classify only one view at a time, because side view might have different features than frontal, and mixing this features might be confusing for a network. However I am not really sure. If something is unclear, please let me know.

[EDIT] Since problem might be hard to understand only by reading, I made some illustrations. Basically I wonder if using second option can help in achieveing better accuracy for the subtle differences like those in gender recognition, especially when face is not visible:

  1. Single detection and classification: enter image description here

  2. Two different classifiers: enter image description here

Img Source

Improve this question
$\endgroup$
1
  • $\begingroup$ I edited the question giving some graphical explanation of what I am asking for. $\endgroup$
    – GKozinski
    Sep 10, 2019 at 8:31

1 Answer 1

Reset to default
1
$\begingroup$

I think your fundamental question is: can convolutional neural networks generalize to objects independent of the view?
The answer is mostly yes (given the dataset contains multiple views of objects). This is evident by looking at the results of various challenges: e.g. COCO object detection challenge results on youtube. You can see that no matter what the view is on the car, pedestrians, the detector is not biased towards any specific viewpoint.

Therefore, one can assume that you can build only one network to perform object detection and another network to perform classification.

If you really want to go even further:
- you can make a small change to the architecture of your detector (I guess you might be using something like SSD, YOLO or Faster-RCNN), in which you make gender classification for every bounding box prediction. If you think about it, it is intuitive because the detector is already doing classification (there is softmax + cross-entropy loss usually), you can just add another term in its tensor output and modify the loss. That way you don't even need another network! It would be much faster and simpler.
- you can predict the pose estimation of the object (and corresponding normals) with respect to the camera to capture the best viewpoint to perform classification.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Yes, you are right. What I was more thinking about (and now I see I did not write it) is that gender classification (and I am also thinking about extending this to age-group classification) can sometimes be very tricky because differences might be very subtle, especially with some weird fashion styles of people. So I thought that maybe by making this proposed two views division it could be easier for network to focus on these subtle details. $\endgroup$
    – GKozinski
    Sep 11, 2019 at 13:00
  • $\begingroup$ And can you please elaborate a bit more on your last point: - you can predict the pose estimation of the object (and corresponding normals) with respect to the camera to capture the best viewpoint to perform classification. What excatly do you mean? Maybe some links? $\endgroup$
    – GKozinski
    Sep 11, 2019 at 13:02
  • $\begingroup$ Yeah I see. You can definitely try it and compare empirically. Also your point might be true in the case if you have little data. If you have enough/a lot of data (20k+ images) then having one architecture will do. $\endgroup$
    – Anuar Y
    Sep 11, 2019 at 22:35
  • $\begingroup$ In terms of pose estimation I meant 2D pose estimation, for example this famous work. This does keypoint detection in the space of the image. One can also do it in 3D but thats' more complicated with monocular cameras. I meant that having the exact 3d Pose from a human, given you have a video you can for example select which frame (to get best viewpoint) to run your 2nd classification network on. $\endgroup$
    – Anuar Y
    Sep 11, 2019 at 22:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .