While reading some explanations of why dense word embeddings work better than sparse word embeddings, the following statement has been given in the chapter Vector Semantics and Embeddings, showing a drawback of sparse word embeddings.

Dense vectors may also do a better job of capturing synonymy. For example, in a sparse vector representation, dimensions for synonyms like car and automobile dimension are distinct and unrelated; sparse vectors may thus fail to capture the similarity between a word with car as a neighbor and a word with automobile as a neighbor.

It says that the dimensions of synonyms may be unrelated and distinct. I am facing difficulty in understanding it.

Can anyone provide me a simple example to understand it by taking some simple dimensions which are unrelated and distinct?

You can consider either documents or (context) words as dimensions for the example.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.