Image recognition service architecture

Question

A question for developers of projects for pattern recognition. How best to organize the architecture of such a service?

At what stage do you conduct logic? (for example, for the recognition of a photo of a male blue jacket, a cascade of queries is performed: "recognizing men" -> "recognizing the jacket" -> "recognizing the color of the jacket.")

Does it make sense to implement all search options within a single neural network or is it better to create a set of individual neuronets that are confined to fairly simple tasks?

Answer 1

That is one of the good example for research. Personally, I prefer to segment out all the desired outputs at once. Then, check the success rate. If you cannot hit the success rate that you desire, you can go for more specific solutions for the specific problem that you face.

However, in general, the localization, segmentation, recognition are implemented in same network and are obtained all-at-once.

Answer 2

I would use a single network:

The essence of the question is whether or not doing all the classification work at once is more efficient than running individual classifiers for each stage.

The recent "You Only Look Once" algorithm ("YOLO") is based on the fact that the convolutional networks can reuse a lot of the interim calculations if you combine them into one. Because of this, they are able to perform real-time object detection on images across thousands of classes.

You can express your hierarchical classifier with YOLO (man, jacket and jacket colour classes). Depending on your needs, you might want to model the jacket colour as a scalar output of an approximate R,G,B value for the colour rather than having named classes for the colours.

This everything-at-once implementation gives you runtime efficiencies for the inference step and much faster training since the classes share common abstractions in the earlier layers of the net.

Details, the YOLO version 2 paper, and a cool demonstration video featuring James Bond are available here: https://pjreddie.com/darknet/yolo/

The paper itself, "YOLO9000", is available on Arxiv: https://arxiv.org/abs/1612.08242