5
$\begingroup$

What does the term "embedding" actually mean?

An embedding is a vector, but is that vector a representation of a word or its meaning? Literature loosely uses the word for both purposes. Which one is actually correct?

Or is there anything like: A word is its meaning itself?

$\endgroup$
5
$\begingroup$

An embedding is a representation of a word that can be used as a proxy for some of its linguistic properties.

The 'human' representation of a word, a sequence of letters and other symbols, is not related at all to its meaning or use in actual text. It only serves as a look-up key into our cognitive language processing facility (however that actually works) which enables us to understand the meaning in the context of its usage. However, a computer system does not have such a facility.

In order to convert the character string into something more usable for language processing, embeddings are created. These are typically vectors describing other words that surround a word in question (as the meaning of a word depends on its context). There are obvious problems with ambiguities (eg is bank the side of a river or a place where you deposit money?), but there are probably ways around that.

So the embedding (a vector) represents the usage of a word, which strongly correlates with its meaning. Because words as character sequences are useless for most sub-symbolic processing, the terms embedding and word are possibly used interchangably.

$\endgroup$
2
$\begingroup$

Although we have had multiple similar questions (see here, here and here) and it seems to me that you focused on word embeddings (probably because you were not aware of the application of embeddings to other contexts), in addition to what is stated in the other answer, it's important to note that the concept of an embedding does not just apply to words. For example, there are also code embeddings (see e.g. code2vec) and graph embeddings (see e.g. this), and there are probably other examples. The linked posts contain answers that explain what an embedding and embedding space generally are, so you may want to read them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.