0
$\begingroup$

I have a bunch of unique full names of users. I made pseudo-physical model to emulate misprints of desktop and mobile users (hence, fatfingering, jumpy fingers, accidentals touches of touch bar etc.)

So, I have pairs like John Snow - joh Snown

I tried first Recurrent networks, LSTM, like some kind of vocabulary to "translate" from bad words to good ones, but it return only known predicted result, and when I try to put unknown last names, it returns wrong results.

I wish to find some patterns in misspelled words, and to predict correct spelling.

Can you please advice some kind of NN to cope with the task, or maybe some contributions in that domain?

P.S. Yes, I know that there exist other AI methods to get things done

P.P.S. This vocabulary is not in English, just in case

UPDATE

LSTM nn works nice with known names and last names endings for new last names. Right now I use 2 different nn, first to correct mistypes, second to determine first, last and middle name.

$\endgroup$
2
  • $\begingroup$ Are you sure there is a pattern in misspelled words? Also, even if there is a pattern, how can a model distinguish between a misspelled word and a correctly spelled word when it confronts an unknown last name? For example like in John Snow - Joh Snown, how do we know for sure that the user's name is not actually Joh Snown? $\endgroup$
    – DKDK
    Dec 6 '21 at 1:22
  • $\begingroup$ @DKDK there are some patterns that can not exist in real language. As for russian, for example, it is very rare to find some combinations of consonants, sequences of letters, sequences of letters in first word if second has some specific words ending, when second is feminine and first have masculine ending etc. because of a grammar of the language. There will happen some faults, but it is a recommendation system. I know some systems that distinguish mistypes with big rule sets, and they work (not perfectly, but even if I reach +5% more correct names that will save time and money) $\endgroup$ Dec 6 '21 at 6:13
0
$\begingroup$

I think it would be a hard task for identifying spell errors in first and last names but you can fine-tune a BERT and use Damerau–Levenshtein distance in your final prediction. for further information you can read the paper below.

Misspelling Correction with Pre-trained Contextual Language Model

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.