0
$\begingroup$

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams.

Why would adding all the possible embeddings, $\binom{10}{2}$ or $\binom{10}{3}$, be "worse" than using 1D-convolutions?

$\endgroup$
0
0
$\begingroup$

N-grams are defined as sets of n contiguous words. We use n-grams because they are more useful than random combinations of words across the sentence. Intuitively, combinations of nearby words have more semantic meaning than combinations of distant words.

Also, using all possible combinations of n embeddings would take much longer, especially since (1D) convolutions are such efficient operations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.