Skip to main content

Questions tagged [skip-gram]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
24 views

Skip-Gram Model description in word2vec explanation article

In his article word2vec Parameter Learning Explained Xin Rong says (page 7): Each output is computed using the same hidden->output matrix: $$ p(w_{c,j} = w_{O.c}|w_I)=y_{c,j}=\frac{exp(u_{c,j})}{\...
Damir Tenishev's user avatar
0 votes
0 answers
16 views

Can 3D convolutions appropriately capture a frozen embedding space?

My project is a strange combination of NLP and Computer Vision. I have datapoints of 3D tensor where each element is a token in an NLP vocabulary. The vocabulary is around 1000 unique "words"...
schmixi's user avatar
0 votes
1 answer
33 views

Why does skip-gram uses linear maps for embedding?

This question is inspired from Lilian Weng's blog here about skip-gram model, where she shows the model as multiplication via 2 matrices $W$ and $W'$ for embedding and word context matrix and $W$ ...
Mahammad Yusifov's user avatar
1 vote
1 answer
783 views

Is my interpretation of the mathematics of the CBOW and Skip-Gram models correct?

I am a mathematics student who is learning NLP, so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations may or may not be right sometimes. Please ...
Robert's user avatar
  • 111
20 votes
2 answers
20k views

What are the main differences between skip-gram and continuous bag of words?

The skip-gram and continuous bag of words (CBOW) are two different types of word2vec models. What are the main differences between them? What are the pros and cons of both methods?
DRV's user avatar
  • 1,703