0
$\begingroup$

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams.

Question 1 Why would adding all the possible $\binom{10}{2}$ or $\binom{10}{3}$ embeddings be "worse" then using 1D-convolutions?

Question 2 Also for each of the $2$-grams and $3$-grams, would you try to learn some large number of $2 \times 2$ filters and $3 \times 3$ filters so that you can convolve it with the word2vec embedding? How do you learn these filters?

$\endgroup$
  • $\begingroup$ Maybe you should post your Question 2 as a separate question. One distinct question per post is usually better. $\endgroup$ – Philip Raeisghasem Apr 13 at 22:05
0
$\begingroup$

I can answer question 1. N-grams are defined as sets of n contiguous words. We use n-grams because they are more useful than random combinations of words across the sentence. Intuitively, combinations of nearby words have more semantic meaning than combinations of distant words.

Also, using all possible combinations of n embeddings would take much longer, especially since (1D) convolutions are such efficient operations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.