0
$\begingroup$

I would like to train a model that serializes a table of nutrition facts into it's values. The tables can vary in form and colour, but always contain the same set of keys (e.g. carbs, fats). Examples for these tables can be found here.

The end goal is to be able to take a picture of such a table and have it's values added to a database.

My initial idea was to train a model on finding subpictures of the individual key/value pairs and then using OCR to find out which value it actually is.

As I am relatively new to ML, I would love to have some ideas about how one could try to build this, so I can do further research on it.

Thanks

Improve this question
$\endgroup$
0
$\begingroup$

Assuming all of the tables will be oriented in similar ways (label and value running horizontally) and that all writing will be printed rather than handwritten, one solution method would be to use an image segmentation method such as edge detection to segregate these horizontal (label, value) pairs and then use a library like Tesseract for OCR.

There are many types of image segmentation methods that may all have value, but if my assumption holds regarding the neat, structured nature of the tables, then I think simple edge detection methods could be sufficient.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Traditional edge detection may not be enough because the tables could have no lines between cells. The last link gave me a good idea about image segregation though. I might try to train a model to detect the various segments and use Tesseract to read the value. $\endgroup$
    – kapuetze
    Jul 16 '20 at 13:14
  • $\begingroup$ Ah, my mistake @kapuetze - I didn't see the examples of tables with no horizontal lines. Still, glad you found something useful. $\endgroup$
    – soitgoes
    Jul 17 '20 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.