There are many, many literary works in the public domain, along with human translations, many of which have entered the public domain as well. (Public domain = easily available)

In order for me to advance my knowledge of e.g. Japanese, I'd like to read texts in Japanese using a yet to be written tool, and when I encounter an unknown word/phrase/sentence, I'd like to just click on a position in the text and be transferred to the corresponding position in the e.g. English translation (or original) of the text in question.

Let's also assume we have access to a dictionary that translates a fair amount of words between those two languages (and there are of course free dictionaries for many language pairs).

What ways are there to use AI toolkits plus some wiring and perhaps scripting/programming, to auto-correlate the positions of two versions of the same text, in two languages? The results do not - and in fact in many cases cannot - be perfect, but they should be roughly correct.

I'm aware that this is still not a straightforwards task, as there are complicating factors like inflection of verbs and other grammatical properties that make the use of dictionary tools much harder. Also, translators will often translate to words that don't have that mapping in any dictionary. Then there is the fact that words aren't delimited by spaces in languages like my example language Japanese - (but if it is easier to work with only space-separated languages like, say, Spanish, or Russian, I'd like to hear answers to this simpler problem as well). Also the order of words and even hole sub-clauses differs from language to language.

A simple, non-AI approximation would be to

  1. figure out at what relative position in the source language text the user clicked (e.g. at the character on position 50.234%)
  2. then go to that same relative position 50.234% in the target language text

This approximation could perhaps be used as the starting point for the AI, which would then use words and dictionaries to make the results more accurate.


You are basically describing the way Google Translate works.

There has been a lot of research in text alignment in the area of multi-lingual corpus linguistics. An early paper (with sourcode) is Gale and Church's A Program for Aligning Sentences in Bilingual Corpora (PDF).

In linguistics these are called parallel texts. On the wikipedia page you will find links to a number of alignment programs which attempt to solve that issue.

  • $\begingroup$ Awesome. Although your answer doesn't spell out a way to implement this, the terms you gave are the nudge I needed to continue my research. $\endgroup$ Mar 10 '19 at 22:51
  • $\begingroup$ There is C source code in the Gale and Church PDF. $\endgroup$ Mar 11 '19 at 9:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.