Questions tagged [skip-gram]
The skip-gram tag has no usage guidance.
5 questions
0
votes
0
answers
45
views
Skip-Gram Model description in word2vec explanation article
In his article word2vec Parameter Learning Explained Xin Rong says (page 7):
Each output is computed using the same hidden->output matrix:
$$ p(w_{c,j} = w_{O.c}|w_I)=y_{c,j}=\frac{exp(u_{c,j})}{\...
- 188
0
votes
0
answers
25
views
Can 3D convolutions appropriately capture a frozen embedding space?
My project is a strange combination of NLP and Computer Vision.
I have datapoints of 3D tensor where each element is a token in an NLP vocabulary. The vocabulary is around 1000 unique "words"...
- 1
0
votes
1
answer
40
views
Why does skip-gram uses linear maps for embedding?
This question is inspired from Lilian Weng's blog here about skip-gram model, where she shows the model as multiplication via 2 matrices $W$ and $W'$ for embedding and word context matrix and $W$ ...
- 113
1
vote
1
answer
846
views
Is my interpretation of the mathematics of the CBOW and Skip-Gram models correct?
I am a mathematics student who is learning NLP, so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations may or may not be right sometimes. Please ...
- 111
20
votes
2
answers
20k
views
What are the main differences between skip-gram and continuous bag of words?
The skip-gram and continuous bag of words (CBOW) are two different types of word2vec models.
What are the main differences between them? What are the pros and cons of both methods?
- 1,763