I have seen the extreme effectivity of embeddings from networks trained on other tasks (such as next token prediction) at bringing semantically similar items in the domain "close" in the space of the vectors of their embeddings. It seems that because of this vector databases are extremely important in production ML settings. I'm wondering if there is any literature investigating why embeddings from neural networks have this quality and why one would actually expect (almost all) semantically linked subjects to be close together in the embedding space, instead of, let's say random clusters or something else. Additionally, literature on what the limits are of embedding similarities, i.e. are there semantically linked things neural networks struggle with?
1 Answer
It seems you're mainly talking about the effectiveness of word embeddings in natural language processing (NLP) tasks and they've been widely studied for several theoretical underpinnings of such effectiveness and limitations. One of the main reasons is the distributional hypothesis of language models in linguistics.
The distributional hypothesis in linguistics is derived from the semantic theory of language usage, i.e. words that are used and occur in the same contexts tend to purport similar meanings... The distributional hypothesis suggests that the more semantically similar two words are, the more distributionally similar they will be in turn, and thus the more that they will tend to occur in similar linguistic contexts... The basic approach is to collect distributional information in high-dimensional vectors, and to define distributional/semantic similarity in terms of vector similarity. Different kinds of similarities can be extracted depending on which type of distributional information is used to collect the vectors: topical similarities can be extracted by populating the vectors with information on which text regions the linguistic items occur in; paradigmatic similarities can be extracted by populating the vectors with information on which other linguistic items the items co-occur with.
The main literature on distributional semantics hypothesis are:
Rieger, Burghard B. (1991). "On Distributed Representations in Word Semantics"
Mikolov et al. (2013). "Efficient estimation of word representations in vector space"
On the other hand, modern DNNs building hierarchical smooth representations of data where lower layers capture simple syntactic features and higher layers capture more abstract semantic features are well suited for producing high dimensional embedding vectors that reflect above various kinds of similarities among linguistic items. Furthermore, the loss functions of supervised and self-supervised learning inherently encourage the network to bring similar vectors closer and push dissimilar vectors apart on a continuous manifold of embedding space.
The main literature on DNNs' technical capabilities for embedding's similarity learning are:
Bengio et al. (2013). "Representational learning: A review and new perspectives"
Vaswani et al. (2017). "Attention is all you need"
J. Devlin et al. (2018). "BERT: Pre-training of deep bidirectional transformers for language understanding"
Lastly, words or linguistic items with multiple meanings can create challenges for embeddings as a single vector. Contextual embeddings such as BERT address this to some extent by providing different embeddings based on context. Embeddings similarities trained on specific domains may not generalize well to other domains. Domain adaptation can help but there're limits to how well embeddings transfer across very different domains. Embedding similarities can also capture and propagate biases present in their training data and have limitations to express certain nuanced or complex semantics such as negation, antonyms, and out-of-vocabulary (OOV) or rare words.
The main literature on embedding similarities limitations are:
Pan & Yang. (2009). "A survey on transfer learning"
Zhao et al. (2017). "Men also like shopping: Reducing gender bias amplification using corpus-level constraints"
Peters et al. (2018). "Deep contextualized word representations"
