1
$\begingroup$

I hope this question is not too broad or general. I have a very large set of images all of which contain text (some have more, some less). All of them have been tagged as containing, say, English text or Korean. I wonder if convolutional neural networks would be a good approach to classify these images as containing English vs. Korean. Or is there any existing literature/method that does this already. Crucially though, I am not interested in "understanding" the text, so this is not an NLP task but, I suppose, a task of classifying orthographies in the images.

$\endgroup$

3 Answers 3

0
$\begingroup$

This sounds like a fairly straightforward task, with low risks. I think the proper term is that you're you are trying to detect the script, which would be either Latin ( "English") or Hangul ("Korean"). The chance is that you end up learning fonts, though.

$\endgroup$
0
$\begingroup$

I think that you can use convolutional neural networks (CNNs) to do what you want, but I think that you will need a lot of training images and the task will be very difficult. You are much better off extracting text from the images using Optical Character Recognition (OCR), partitioning the extracted text into individual words, and then searching if the words are found in an English dictionary or a Korean dictionary (or both?). This approach isn't perfect, but I bet it will perform better than directly applying CNNs directly to images.

$\endgroup$
0
$\begingroup$

The role of the CNN in this would basically just be to grab the text you want. YOLO is good for that. Fast and lightweight and easy to train. It wouldn't help much in actually reading the text though. A way to do something like this would be to use a neural net for segmentation, training it to draw boxes around text, then you could take the content of those boxes and feed them to an OCR algorithm like Tesseract, and once you have text, then there are NLP tutorials all over the internet on how to do language detection. Here are a couple to start.

https://www.johnsnowlabs.com/how-to-detect-languages-with-python-a-comprehensive-guide/ https://towardsdatascience.com/how-to-detect-and-translate-languages-for-nlp-project-dfd52af0c3b5

It really doesn't matter that you aren't trying to "understand" the text. NLP libraries are still where it's at if you are trying to perform a language detection task. Trying to treat language detection as an image classification task is like trying to infer the speed limit on a highway by reading the heat signatures on the pavement. Sure, faster cars will transfer more heat, but there is so much entropy involved here that this is never going to be the best option for this task. Better to extract the thing you want, the words, and use models designed to deal with words.

Basic breakdown for this sort of thing in general would be:

  1. Use a neural network to isolate text from an image.
  2. Use OCR to convert that image text into computer readable text
  3. Feed that text to an NLP library to detect the language
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .