How do I identify the number and type of objects in the same picture?

Question

I need to identify the number and type of all objects in a picture, so there can be multiple objects of the same type.

For example, I have a picture with $10$ animals, and I want my program to tell me that, on the picture, I have $3$ elephants, $2$ cats and $5$ dogs. However, I do not need the detection of the location of the objects. All I need is the information on the number of objects of each class, without their possible locations.

I wanted to ask you guys for help in defining the type of problem I am dealing with and maybe some suggestions about where to start looking for a solution. It would be nice if you could point out some directions, algorithms or network architectures to solve the problem described below.

Answer 1

Your problem might involve a combination of object recognition (or object classification), which is the problem of determining which objects are in an image, and object detection (which could also be called object localization), which is the task of locating a specific object in the image.

A naive approach to solve your problem would then be to first locate the position of all possible objects in the image (object detection), then, for each these positions, perform object recognition (or classification). Then you would count the number of detected and classified objects for each possible class.

There are several models that can be used to solve your problem.

The YOLO (You Only Look Once) model (introduced in the paper You Only Look Once: Unified, Real-Time Object Detection) (2016) can be used to detect multiple objects in an image. See the official YOLO implementation, which is part of the Darknet framework. Have also a look at this article, which explains how to use the official implementation of YOLO.

You could also use the SSD model (introduced in the paper Single Shot MultiBox Detector). There is also R-CNN (Regions with CNN features), Fast R-CNN or Faster R-CNN, but these should perform worse than both YOLO and SSD.

So, even if you do not care about the locations, YOLO (and other mentioned methods) can give them to you, in addition to the number and type of objects in the image.