A homograph - is a word that shares the same written form as another word but has a different meaning.

They can be even different parts of speech. For example:

  1. close(verb) - close(adverb)
  2. lead(verb) - lead(noun)
  3. wind(noun) - wind(verb)

And there is rather a big list https://en.wikipedia.org/wiki/List_of_English_homographs.

As far as I understand, after processing the text data in any conventional way, lemmatization, building an embedding, these words, despite having different meaning, and appearing in different contexts, would be absolutely the same for the algorithm, and in the end we would get some averaged context between two or more meainings of the word. And this embedding would be meaningless.

How is this problem treated or these words are regarded to be too rare to have a significant impact on the quality of resulting embedding?

I would appreciate comments and references to the papers or sources

  • $\begingroup$ I have exactly the same question and wonder if after these 7 months with no replies the original questioner has found some reference elsewhere. Thanks. $\endgroup$ Mar 26, 2021 at 8:21
  • $\begingroup$ Actually, contextual embeddings - stackoverflow.com/questions/62272056/…, arxiv.org/pdf/2003.07278.pdf - take each token not by itself alone, but in the surrounding context. These words are likely appear in surrounding of rather different words - so the resulting embedding for them would be different. $\endgroup$ Mar 26, 2021 at 10:30


You must log in to answer this question.