Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
What does the term "embedding" actually mean?
An embedding is a vector, but is that vector a representation of a word or its meaning? Literature loosely uses the word for both purposes. Which one is actually correct?
Or is there anything like: A word is its meaning itself?
An embedding is a representation of a word that can be used as a proxy for some of its linguistic properties.
The 'human' representation of a word, a sequence of letters and other symbols, is not related at all to its meaning or use in actual text. It only serves as a look-up key into our cognitive language processing facility (however that actually works) which enables us to understand the meaning in the context of its usage. However, a computer system does not have such a facility.
In order to convert the character string into something more usable for language processing, embeddings are created. These are typically vectors describing other words that surround a word in question (as the meaning of a word depends on its context). There are obvious problems with ambiguities (eg is bank the side of a river or a place where you deposit money?), but there are probably ways around that.
So the embedding (a vector) represents the usage of a word, which strongly correlates with its meaning. Because words as character sequences are useless for most sub-symbolic processing, the terms embedding and word are possibly used interchangably.
Although we have had multiple similar questions (see here, here and here) and it seems to me that you focused on word embeddings (probably because you were not aware of the application of embeddings to other contexts), in addition to what is stated in the other answer, it's important to note that the concept of an embedding does not just apply to words. For example, there are also code embeddings (see e.g. code2vec) and graph embeddings (see e.g. this), and there are probably other examples. The linked posts contain answers that explain what an embedding and embedding space generally are, so you may want to read them.
