I have images of questions from the domain of mathematics, where the image can be a mixture of the English language and mathematical formulas. I want to build and train an OCR model like Harvard NLP's Im2Markup, but the problem is that, even if I am able to replicate and train such a model, how will I be able to extract from MY TYPE OF IMAGES? Because the im2latex-100k training data has pure equations of mathematical formulas and my image has Texts and Formulas.
Let us even say that by using MathPix, I am able to extract the Latex for a lot of images say 100000 or so, how can I then approach the training model?
One approach is that I try to classify whether a question has mathematical formulas or not using a classification model. But even if I classify if there is a formula presented, how am I able to extract the latex along with the Text?