Best approach for 2D Grid Image Segmentation

I'm working on a project where I need to extract text from grocery discount flyers like the Costco announcement below (retrieved in a random google search, Costco is not the deal here):

If I just run OCR (like with Tesseract in python):

import cv2
import pytesseract
text = pytesseract.image_to_string(img)
print(text)


I get:

variety pack packet

ere $$12.99 i rom hagst 31026 2012 > > Je laa > + a > > Wrigley’s Excel Gum variety > > Backol 24 > >$$13.79 fom agus 26.202

OFF

Solon Extra virgin olive oil [...]

Which is a lot noisy.

My guess is that splitting the image to its base squares enchances the recognition.

However, I'm confused on how to do it. I can classify images using a CNN, but am not sure about object recognition.

Should I have a sliding window and train several "grid box" objects on a generic CNN and then provide this window data to be classified? How to adapt to distinct object window sizes?

This is a really cool problem. You already have a working model here are a few different ways of going forward with the project.

Training the model to detect boxes followed by extracting the text from each box seems like a very smart direction to move in. This paper talks about the former. automatic image segmentation and edge detection Good luck!

• Thank you for the directions! I'll look at them and return the check answer as soon as I've figured it out. Nov 17, 2019 at 23:58