1
$\begingroup$

While reading some explanations of why dense word embeddings work better than sparse word embeddings, the following statement has been given in the chapter Vector Semantics and Embeddings, showing a drawback of sparse word embeddings.

Dense vectors may also do a better job of capturing synonymy. For example, in a sparse vector representation, dimensions for synonyms like car and automobile dimension are distinct and unrelated; sparse vectors may thus fail to capture the similarity between a word with car as a neighbor and a word with automobile as a neighbor.

It says that the dimensions of synonyms may be unrelated and distinct. I am facing difficulty in understanding it.

Can anyone provide me a simple example to understand it by taking some simple dimensions which are unrelated and distinct?

You can consider either documents or (context) words as dimensions for the example.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.