3
$\begingroup$

Recent advances in Deeplearning and dedicated hardware has made it possible to detect images with a much better accuracy than ever. Neural networks are the gold standard for computer vision application and are used widely in the industry, for example for internet search engines and autonomous cars. In real life problems, the image contains of regions with different objects. It is not enough to only identify the picture but elements of the picture.

A while ago an alternative to the well known sliding window algorithm was described in the literature, called Region Proposal Networks. It is basically a convolution neural network which was extended by a region vector.

Problem that I am trying to solve:

In a given video frame, I want to pick some region of interests (literally), and perform classification on those regions.

How is it currently implemented

  1. Capture the video frame
  2. Split the video frame into multiple images each representing a region of interest
  3. Perform image classification(inference) on each of the image (corresponding to a part of the frame)
  4. Aggregate the results of #3

Problem with the current approach

Multiple inferences per frame.

Question

I am looking for a solution where I specify the locations of interest in a frame, and inference task, be it object detection (or) image classification, is performed only on those regions.Can you please point to me the references which I need to study (or) use to do this.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Is your requirement to supply your own "region of interest" data fixed? That would rule out architectures like YOLO, which do very close to what you want overall, but internally decide on locations. $\endgroup$
    – Neil Slater
    Aug 24, 2018 at 9:53

2 Answers 2

Reset to default
0
$\begingroup$

There are many different problems in computer vision. Four of them are well described by the top image here:

  • Classification: Given an image, say what is on it (a single thing)
  • Classification+Localization: Given an image, say what is on it and draw an axis-aligned bounding box (AABB) around it
  • Object detection: Given an image, draw AABBs around every object and classify those objects
  • Semantic segmentation: See Survey of semantic segmentation
  • Instance segmentatation: Like semantic segmentation, but if there are multiple cats then they should be recognized as different objects.

Your question seems to be about object detection. The relevant papers here are:

If you actually already have the regions, then you can simply perform classification on them. When you pad / scale / crop them, you can batch-predict them.

Improve this answer
$\endgroup$
0
$\begingroup$

It might be easier to approach as an object segmentation problem which identifies multiple objects in a given image/frame. There are lots of examples of you do a search using “object segmentation” as keyword.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Object segmentation seems to be an umbrella term that can mean either semantic segmentation or instance segmentation. Which one do you refer to? $\endgroup$
    – HelloGoodbye
    Jan 2 at 1:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .