I'm working on a project where I need to extract text from grocery discount flyers like the Costco announcement below (retrieved in a random google search, Costco is not the deal here):

example sample image

If I just run OCR (like with Tesseract in python):

import cv2
import pytesseract
img = cv2.imread('costco.jpg')
text = pytesseract.image_to_string(img)

I get:

Cadbury Chocolate

variety pack packet

ere $12.99 i rom hagst 31026 2012 > > Je laa > + a > > Wrigley’s Excel Gum variety > > Backol 24 > > $13.79 fom agus 26.202


Solon Extra virgin olive oil [...]

Which is a lot noisy.

My guess is that splitting the image to its base squares enchances the recognition.

However, I'm confused on how to do it. I can classify images using a CNN, but am not sure about object recognition.

Should I have a sliding window and train several "grid box" objects on a generic CNN and then provide this window data to be classified? How to adapt to distinct object window sizes?


This is a really cool problem. You already have a working model here are a few different ways of going forward with the project.

Training the model to detect boxes followed by extracting the text from each box seems like a very smart direction to move in. This paper talks about the former. automatic image segmentation and edge detection Good luck!

  • $\begingroup$ Thank you for the directions! I'll look at them and return the check answer as soon as I've figured it out. $\endgroup$ Nov 17 '19 at 23:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.