I am very new to AI and ML alike. I am using google's OCR to extract text from image like receipts and invoice. Can someone please guide me to which techniques are used to make sense of the text. Like I would like to extract date, name of business, address, total amount, etc. If someone can please direct me to right set of algorithm industry uses for machine learning before marking this question "too broad", will be great.

I know this question is too broad, but there are no one to one answers for these questions. I will delete this question once answered and if marked not useful to keep forum clean.

  • 2
    $\begingroup$ Welcome to AI! Let's see what the community thinks about the broadness, but I think this is a useful, basic question. $\endgroup$ – DukeZhou Nov 21 '17 at 22:27

Last semester I, along with my team, made a project on OCR.
Note: I am assuming the data set for your pictures has white background with black (or some other dark text) on it.
These are the overview steps we followed:
enter image description here
Pre-processing includes grayscale conversion, noise reduction, binarization and skew detection.
Next step was segmentation. This process extracts the individual characters from the image. Histogram taken along the y-axis divided the image into lines. This is followed by histogram along the x-axis which divided them into words and further into characters. At the end of the step, we used Savgol filter to smooth the curves of the histogram.

Next step was feature extraction. This is the most important step. The accuracy of your code depends on how well your features are.
We used the following features:

  • Crossing: Counting number of transitions between foreground and background. We used two diagonal lines, two horizontal and one vertical line. You can used any number you want.
  • Zoning: Whole character region is divided into 16 zones, and density of each zone is measured.
  • Projection Histogram: Each character has unique (almost) vertical and horizontal histogram signature.
  • Other features include number of endpoints in the character, number of loops and horizontal/vertical line count.

We used three different classification algorithms for our project. They were KNN (K-Nearest Neighbours), Artificial Neural Network (ANN) and Extra Tree classification. Their F1 score was 0.84, 0.82 and 0.77 respectively.
For training, you will need to find datasets. Many data sets for OCR was available online. Make sure you are using good ones.

| improve this answer | |
  • $\begingroup$ Remember we are talking about finding relation between data we extracted from a document. $\endgroup$ – Abhay Naik Nov 24 '17 at 11:32

An interesting question, I think the algorithm used for OCR is "Logistic Regression" or "Decision Tree" in multiple steps.

The steps can be

  1. Image Classification - In this step, the images are classified into "with or without" text.
  2. Text Detection - In this step, the images with text are taken divided into blocks and the blocks are classified into "with or without" text.
  3. Character Detection - In this step, the blocks with text are taken and divided into smaller boxes of single characters and compare with a database of characters.

The database is built using the crowdsourced "captcha" project.

| improve this answer | |
  • $\begingroup$ Can you please explain with an example. $\endgroup$ – Abhay Naik Nov 24 '17 at 11:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.