Word2Vec model does not use any neural network. It uses logistic regression only.

Consider the following paragraph from p:18 of Vector Semantics and Embeddings

We’ll see how to do neural networks in the next chapter, but word2vec is a much simpler model than the neural network language model, in two ways. First,word2vec simplifies the task (making it binary classification instead of word prediction). Second, word2vec simplifies the architecture (training a logistic regression classifier instead of a multi-layer neural network with hidden layers that demand more sophisticated training algorithms). The intuition of skip-gram is:

  1. Treat the target word and a neighboring context word as positive examples.

  2. Randomly sample other words in the lexicon to get negative samples.

  3. Use logistic regression to train a classifier to distinguish those two cases.

  4. Use the learned weights as the embeddings.

But, why it is called a neural model then? Is there any version of Word2Vec tat do use neural networks in it?

  • $\begingroup$ Please write a comment for downvote so that I can improve my question. $\endgroup$ – hanugm Jun 11 at 6:42
  • 1
    $\begingroup$ The Wikipedia article for word2vec en.wikipedia.org/wiki/Word2vec says that a neural network is indeed used: "The word2vec algorithm uses a neural network model to learn word associations from a large corpus of text." What made you think that a neural network isn't used? $\endgroup$ – The Pointer Jun 11 at 9:28
  • $\begingroup$ @ThePointer But the textbook I read is saying differently. It is using logistic regression only. $\endgroup$ – hanugm Jun 11 at 9:29
  • $\begingroup$ Please provide a reference to your textbook and where it is implied that word2vec only uses logistic regression. $\endgroup$ – The Pointer Jun 11 at 9:30
  • 1
    $\begingroup$ This is beyond my level of knowledge at the moment, so it's best to leave it to someone else. $\endgroup$ – The Pointer Jun 11 at 9:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.