3
$\begingroup$

The boy lifted the bat and hit the ball.

In the above sentence, the noun "bat" means the wooden stick. It does not mean bat, the flying mammal, which is also a noun. Using NLP libraries to find the noun version of the definition would still be ambiguous.

How would one go about writing an algorithm that gets the exact definition, given a word, and the sentence it is used in?

I was thinking you could use word2vec, then use autoextend https://arxiv.org/pdf/1507.01127.pdf to differentiate between 2 different lexemes e.g. bat (animal) and bat (wooden stick).

Then the closest cosine distance between the dictionary definition and any of the words of the sentence might indicate the correct definition.

Does this sound correct?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I'd suggest BERT for this. It is essentially a word-embedding model that uses at local context to determine the appropriate embedding for each word. This means it would assign "bat" a different embedding in a sentence containing "hit the ball" vs. a sentence containing "flies and eats bugs". On top of that, Google has released a number of pre-trained versions of BERT, which can be used directly without additional training (depending on your task of course). BERT as a service is great if you just want embeddings. The Python transformers library makes it exceedingly simple to incorporate BERT into your task-specific model.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Could you please elaborate on how to utilise BERT? Which functions to use to achieve the task outlined? From what I've read, BERT isn't good for cosine similarity. So how can this be used? $\endgroup$
    – Yoker
    Feb 28, 2021 at 3:12
  • $\begingroup$ Sorry I think I misunderstood your use-case the first time around. It seems you want to match tokens in context to dictionary definitions? This is a non-trivial problem. You're right that BERT may not be the best choice because the embeddings for the same word change depending on context. Autoextend may work. You may also use BERT to average the embeddings for the words of the same definition on a corpus where the ground truth meanings are known and annotated, then for new sentences compute the cosine sim between the new occurrence of the word and the averaged embedding. $\endgroup$
    – primussucks
    Mar 2, 2021 at 16:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .