Skip to main content
1 of 2

1D convolutions, word2vec, and $n$-grams

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams.

Question 1 Why would adding all the possible $\binom{10}{2}$ or $\binom{10}{3}$ embeddings be "worse" then using 1D-convolutions?

Question 2 Also for each of the $2$-grams and $3$-grams, would you try to learn some large number of $2 \times 2$ filters and $3 \times 3$ filters so that you can convolve it with the word2vec embedding? How do you learn these filters?