1
$\begingroup$

this is my first question on AI Stack Exchange. I am a mathematics student who is learning NLP so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations may or may not be right sometimes. Please correct me if any of them are incorrect or do not make sense.

So I have learned CBOW and Skip-Gram models. I think I have understood the CBOW model, and here is my interpretation: First, we fix a number of neighbors of the unknown center word which we would like to predict; let the number be $m$. We then input the original characteristic vectors (vectors of zeros and ones only) of those $2m$ context words. By multiplying those vectors by a matrix, we obtain $2m$ new vectors. Next, we take the average of those $2m$ vectors and this is our hidden layer, namely $v$. We finally multiply $v$ with another matrix, and that is the "empirical" result.

I tried to follow the logic to Skip-Gram similarly, but I have been stuck. I understand that Skip-Gram is kind of a "reversal" of CBOW, but the specific steps have given me a hard time. So in Skip-Gram, we only have a center word, and based upon that we are trying to predict $2m$ context words. By similar steps, we obtain a hidden layer, which is again a vector. The final process also involves multiplication with a matrix, but I don't know how we can get $2m$ new vectors based upon one, unless we have $2m$ different matrices?

I appreciate any insights or answers for my question. Thanks!

$\endgroup$
  • $\begingroup$ Welcome to our community. $\endgroup$ – ddaedalus Oct 18 at 17:52
1
$\begingroup$

The following figure from this article can be helpful:

enter image description here

This figure represents "Skip-Gram model structure. Current center word is 'passes'".

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.