Questions tagged [word2vec]

This tag is for questions relating to the language and models that create them.

11 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
23 views

Why does all of NLP literature use noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
2
votes
0answers
75 views

How to perform unsupervised anomaly detection from log file with mostly textual data?

I have a log file of the format, Index, Date, Timestamp, Module, App, Context, Session, Verbosity level, Description The log file can be considered as a master log, which consists of individual ...
2
votes
0answers
31 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
1
vote
0answers
18 views

Creating Text Features using word2vec

My task is to classify some texts. I have used word2vec to represent text words and I pass them to an LSTM as input. Taking into account that texts do not contain the same number of words, is it a ...
0
votes
0answers
12 views

What is the right way to set the dimension of the word representation in SkipGram word2vec?

I know that in word2vec each word has two word representations; one for the center word and one for the context word. What is the right way to set the dimension of the center word using SkipGram, ...
0
votes
0answers
19 views

Which word representation should I use in word2vec?

I know that in word2vec each word has two word representations; one for center word and one for context word. In NLP, when I want to set the words of a text as an input of an LSTM/RNN using word2vec, ...
0
votes
0answers
25 views

How can I combine word2vec with tf-idf to have concatenated features?

I want to develop a focused crawler using deep reinforcement learning and a priority queue that will work as the crawler frontier. I reckon using x = (state, action)...
0
votes
0answers
11 views

Why word embedding such as word2vec is not used as the output layer of a seq2seq decoder?

It would make sense to make the decoder predict a smaller embedding vector instead of softmax over a large dictionary. The word having the most cosine similarity with the output embedding could be ...
0
votes
0answers
15 views

Confusion on how skip gram implementation is formulated

I'm using this source to understand the skip gram model. Let's say the context size is $4$ ($2$ context words on each side of the target word). This image illustrates how training examples are ...
0
votes
0answers
20 views

Understanding how continuous bag of words method learns embedded representations

I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. First, regarding points 1 to 6 - here's my understanding: If we have a vocabulary $V$, the naive way to ...
-2
votes
1answer
111 views

I need a word database… Any qualities I should look for?

I need a word database to train from. I found a word2vec JS word vector database, but I need a method to teach it which words go in which patterns. Please note that I am not asking to have you ...