The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [word-embedding]

For questions related to word embeddings, which are vector representations of words.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Why word embedding such as word2vec is not used as the output layer of a seq2seq decoder?

It would make sense to make the decoder predict a smaller embedding vector instead of softmax over a large dictionary. The word having the most cosine similarity with the output embedding could be ...
1
vote
2answers
43 views

Why is embedding important in NLP, and how does autoencoder work?

People say embedding is necessary in NLP because if using just the word indices, the efficiency is not high as similar words are supposed to be related to each other. However, I still don't truly get ...
2
votes
0answers
16 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...
0
votes
1answer
22 views

Real time ticket similarity

I'm dealing with a "ticket similarity task". Every time new tickets arrive at the help desk (customer service), I need to compare them and find out about similar ones. In this way, once the operator ...
1
vote
0answers
21 views

Reference request: one-hot encoding outperforming random orthogonal encoding

I experimented with a CNN operating on texts encoded as sequences of character vectors, where characters are encoded as one-hot vectors in one embedding and as random unit length pairwise orthogonal ...
2
votes
1answer
51 views

How does Continuous Bag of Words ensure that similar words are encoded as similar embeddings?

This is related to my earlier question, which I'm trying to break down into parts (this being the first). I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. ...
0
votes
0answers
19 views

Understanding how continuous bag of words method learns embedded representations

I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. First, regarding points 1 to 6 - here's my understanding: If we have a vocabulary $V$, the naive way to ...
2
votes
2answers
37 views

Can ELMO embeddings be used to find the n most similar sentences?

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can ELMO embeddings be used to train a ...
2
votes
0answers
28 views

How does FastText support online learning?

I'm using FastText pre-trained-embedding for tackling a classification task, but I saw it supports also online training (incremental training) for adding domain-specific corpus. How does it work? ...
2
votes
1answer
690 views

Adding BERT embeddings in LSTM embedding Layer

I am planning to use BERT embeddings in the LSTM embedding layer instead of the usual Word2vec/Glove Embeddings. What are the possible ways to do that?
3
votes
1answer
49 views

Will BERT embedding be always same for a given document when used as a feature extractor

When we use BERT embeddings for a classification task, would we get different embeddings everytime we pass the same text through the BERT architecture? If yes, is it the right way to use the ...
0
votes
0answers
22 views

Multiple embedding layers?

How would one go about inputting multiple high dimensionality categorical columns using TensorFlow's Embedding Feature Columns? Does that even make sense to do? For example: for a car price predictor,...
1
vote
0answers
17 views

Creating a zero element in embedding space

I have some variable length input vectors for my own use case of a 'stylistic transfer'-esque process, and I am wondering if anyone knows of a way to engineer an input that maps to a 0 element in ...
0
votes
0answers
23 views

What is the relation between neural embedding and neural code?

Lets consider knowledge graph and operations on it. There are notions of neural embedding and neural coding for it. What is the relation between neural embedding and neural code? Is neural coding a ...
0
votes
0answers
12 views

Embedding Gensim fast-text

Would you suggest to train my own Fast-text embedding using the Gensim library despite i have 1800 sentences and 2k vocabulary length? Don't you think there are too few words? or is there not a ...
2
votes
2answers
54 views

Do we have cross-language vector space for word embedding?

Do we have cross-language vector space for word embedding? When measure similarity for apple/Pomme/mela/Lacus/苹果/りんご, they should be the same If would be great if there's available internet service ...
1
vote
1answer
47 views

How can we create a vector space where word spelling and pronunciation can be easily compared?

In natural language processing, we can convert words to vectors (or word embeddings). In this vector space, we can measure the similarity between these word embeddings. How can we create a vector ...
3
votes
0answers
20 views

Why does all of NLP literature use noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
1
vote
1answer
47 views

Skip-Gram Model Training

Suppose we want to predict context words $w_{i-h}, \dots, w_{i+h}$ given a target word $w_i$ for a window size $h$ around the target word $w_i$. We can represent this as: $$p(w_{i-h}, \dots, w_{i+h}|...
0
votes
1answer
46 views

What do the vectors of the center and outside word look like in word2vec?

In word2vec, the task is to learn to predict which words are most likely to be near each other in some long corpus of text. For each word $c$ in the corpus, the model outputs the probability ...
0
votes
0answers
37 views

What is the right way to convolve over word embeddings?

I have two word embeddings $w_1$ and $w_2$ with dimension 100 as input to a convolutional neural network. It should learn the similarity between these two words. I am now concerned with the applied ...
1
vote
0answers
38 views

How could I learn tree paths given word embeddings?

I need to map from a vector space representation onto a tree structure. A possible solution: given a word vector as input, produce a path in the tree from the root down to the node that most closely ...
3
votes
2answers
104 views

How should the output layer of an LSTM be when the output are word embeddings?

I'm having trouble grasping how to output word embeddings from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot ...
4
votes
1answer
44 views

Do individual dimensions in vector space have meaning?

Word2vec assigns an N-dimensional vector to given words (which can be considered a form of dimensionality reduction). It turns out that, at least with a number of canonical examples, vector ...
-2
votes
1answer
106 views

I need a word database… Any qualities I should look for?

I need a word database to train from. I found a word2vec JS word vector database, but I need a method to teach it which words go in which patterns. Please note that I am not asking to have you ...
2
votes
2answers
61 views

Intuition on how word embeddings bring information to a network

How is it that word embedding layer (say word2vec) brings more insights to the network compared to a simple one hot encoded layer? I understand how word embedding carry some semantic meaning, but it ...
1
vote
1answer
70 views

How is the word embedding represented in the paper “Recurrent neural network based language model”?

I'm reading "Recurrent neural network based language model" of Mikolov et al. (2010). Although the article is straight forward, I'm not sure how word embedding $w(t)$ is obtained: The reason I wonder ...
-2
votes
1answer
42 views

Over-exposure of certain items in content based recommendation engine

I'm working on a content based recommendation engine for ebooks. I create document vectors with 300 features for every ebook using a word2vec model trained on google news and determine recommendations ...
2
votes
0answers
67 views

How can we build word embeddings for a language? [closed]

Which algorithms are there to create word embeddings for a given language?