Skip to main content
Artificial Intelligence

Return to Question

Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
deleted 262 characters in body; edited tags; edited title (comment edited Dec 4, 2020 at 10:20)
Source Link
nbro
nbro
  • 41.4k
  • 12
  • 114
  • 205

Why would adding all the possible embeddings be "worse" than using 1D convolutions, word2vec, and $n$-gramsconvolutions?

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams.

Question 1 Why would adding all the possible $\binom{10}{2}$ or $\binom{10}{3}$ embeddings be "worse" then using 1D-convolutions?

Question 2 Also for each of the $2$-grams and $3$-grams, would you try to learn some large number of $2 \times 2$ filters and $3 \times 3$ filters so that you can convolve it with the word2vec embedding? How do you learn these filters?

Why would adding all the possible embeddings, $\binom{10}{2}$ or $\binom{10}{3}$, be "worse" than using 1D-convolutions?

1D convolutions, word2vec, and $n$-grams

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams.

Question 1 Why would adding all the possible $\binom{10}{2}$ or $\binom{10}{3}$ embeddings be "worse" then using 1D-convolutions?

Question 2 Also for each of the $2$-grams and $3$-grams, would you try to learn some large number of $2 \times 2$ filters and $3 \times 3$ filters so that you can convolve it with the word2vec embedding? How do you learn these filters?

Why would adding all the possible embeddings be "worse" than using 1D-convolutions?

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams.

Why would adding all the possible embeddings, $\binom{10}{2}$ or $\binom{10}{3}$, be "worse" than using 1D-convolutions?

Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Source Link
aiguy123
aiguy123
  • 51
  • 4

1D convolutions, word2vec, and $n$-grams

Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams.

Question 1 Why would adding all the possible $\binom{10}{2}$ or $\binom{10}{3}$ embeddings be "worse" then using 1D-convolutions?

Question 2 Also for each of the $2$-grams and $3$-grams, would you try to learn some large number of $2 \times 2$ filters and $3 \times 3$ filters so that you can convolve it with the word2vec embedding? How do you learn these filters?