Cropping image using ML?

Question

Is there a way to train AI to find aspecific line or symbol in a image and crop it?

OpenCV scripts finds a face and crops it: how can I add my annotations?

Lest say I have a image like this:

+------+ | * *| | | | * *| | | +------+

I want it to find the * and crop it.

Answer 1

Yes, this is possible.

There is actually a pretty easy way that doesn't even require machine learning and can be implemented with a small amount of code. You just use a framework for image processing (e.g. PIL for Python), find the marks by going over your image with an appropriate filter and use the implemented crop function, that the framework hopefully provides (or write i yourself with 2 nested for loops copying the area). Your filter will react strongest in the areas of your image that look like the mark you defined. This could actually be implemented like a single layer conv+RELU layer in a CNN (see this introduction for details about conv and RELU layers).

Because you are asking explicitly about machine learning, I will also give you the hard version to solve this problem using a full CNN to identify the bounding box. Once you have found the bounding box, the actual cropping can be done like in the example above and is not part of the CNN. You could call it post processing if you like.

The technique you are looking for is called Object Localization and Detection. The linked article should tell you all the details you need. You basically build and train your CNN to localize and identify interesting objects in your image. The easy way is to train your CNN to identify the 4 marks individually and return the coordinates of the 4 areas with the highest likelihood to contain your marks. You calculate the center of those 4 individual points and feed it to your cropping function. The hard way is to find the smallest bounding box containing all 4 marks. This is a much harder problem, because the neural network needs to learn that it should ignore everything except those 4 marks during detection, although they will be a small portion of the bounding box. It can be done, but looking for the 4 marks individually would be much easier.

Which approach is best for you? Depends on your goal. If you need a robust and efficient algorithm for this problem, forget about ML and implement it with the straight forward approach I described first. No need for fancy learning CNNs. You can implement this with a few dozen lines of code.

If this is a research or training project to study ML, take my second approach. The easy and the hard way are both fine, the hard one obviously more challenging for your CNN. If you are new to CNN, I recommend following this Stanford Course. It teaches you all you need for your project in 25 to 30 hours (excluding homework).

---

Edit: Concerning your question, how do you add your own marks. I would simply do it using with the same framework you use for cropping. Your program needs some kind of image manipulation capabilities and this will allow pixel manipulation as well (at least the framework PIL which I suggested has this capability). Just draw your mark using this method. You can also copy a sample mark with a transparent background using a tool like Gimp manually, if you prefer that.

Answer 2

If you are so sure that ML is the way to go ahead, then try following.

1. Tensorflow code for object detection

2. YOLO This is especially for real time object detection.

You can train these model for your object in question & see the results.There are some other localization models that you can try.