Questions tagged [word-embedding]

For questions related to word embeddings, which are vector representations of words.

Filter by
Sorted by
Tagged with
2
votes
1answer
42 views

How does Continuous Bag of Words ensure that similar words are encoded as similar embeddings?

This is related to my earlier question, which I'm trying to break down into parts (this being the first). I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. ...
0
votes
0answers
19 views

Understanding how continuous bag of words method learns embedded representations

I'm reading notes on word vectors here. Specifically, I'm referring to section 4.2 on page 7. First, regarding points 1 to 6 - here's my understanding: If we have a vocabulary $V$, the naive way to ...
2
votes
2answers
32 views

Can ELMO embeddings be used to find the n most similar sentences?

Assume I have a list of sentences, which is just a list of strings. I need a way of comparing some input string against those sentences to find the most similar. Can ELMO embeddings be used to train a ...
2
votes
0answers
25 views

How does FastText support online learning?

I'm using FastText pre-trained-embedding for tackling a classification task, but I saw it supports also online training (incremental training) for adding domain-specific corpus. How does it work? ...
2
votes
1answer
230 views

Adding BERT embeddings in LSTM embedding Layer

I am planning to use BERT embeddings in the LSTM embedding layer instead of the usual Word2vec/Glove Embeddings. What are the possible ways to do that?
3
votes
1answer
41 views

Will BERT embedding be always same for a given document when used as a feature extractor

When we use BERT embeddings for a classification task, would we get different embeddings everytime we pass the same text through the BERT architecture? If yes, is it the right way to use the ...
0
votes
0answers
19 views

Multiple embedding layers?

How would one go about inputting multiple high dimensionality categorical columns using TensorFlow's Embedding Feature Columns? Does that even make sense to do? For example: for a car price predictor,...
1
vote
0answers
17 views

Creating a zero element in embedding space

I have some variable length input vectors for my own use case of a 'stylistic transfer'-esque process, and I am wondering if anyone knows of a way to engineer an input that maps to a 0 element in ...
0
votes
0answers
23 views

What is the relation between neural embedding and neural code?

Lets consider knowledge graph and operations on it. There are notions of neural embedding and neural coding for it. What is the relation between neural embedding and neural code? Is neural coding a ...
0
votes
0answers
11 views

Embedding Gensim fast-text

Would you suggest to train my own Fast-text embedding using the Gensim library despite i have 1800 sentences and 2k vocabulary length? Don't you think there are too few words? or is there not a ...
2
votes
2answers
53 views

Do we have cross-language vector space for word embedding?

Do we have cross-language vector space for word embedding? When measure similarity for apple/Pomme/mela/Lacus/苹果/りんご, they should be the same If would be great if there's available internet service ...
1
vote
1answer
46 views

How can we create a vector space where word spelling and pronunciation can be easily compared?

In natural language processing, we can convert words to vectors (or word embeddings). In this vector space, we can measure the similarity between these word embeddings. How can we create a vector ...
3
votes
0answers
19 views

Why does all of NLP literature use noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
1
vote
1answer
46 views

Skip-Gram Model Training

Suppose we want to predict context words $w_{i-h}, \dots, w_{i+h}$ given a target word $w_i$ for a window size $h$ around the target word $w_i$. We can represent this as: $$p(w_{i-h}, \dots, w_{i+h}|...
0
votes
1answer
39 views

What do the vectors of the center and outside word look like in word2vec?

In word2vec, the task is to learn to predict which words are most likely to be near each other in some long corpus of text. For each word $c$ in the corpus, the model outputs the probability ...
0
votes
0answers
34 views

What is the right way to convolve over word embeddings?

I have two word embeddings $w_1$ and $w_2$ with dimension 100 as input to a convolutional neural network. It should learn the similarity between these two words. I am now concerned with the applied ...
1
vote
0answers
36 views

How could I learn tree paths given word embeddings?

I need to map from a vector space representation onto a tree structure. A possible solution: given a word vector as input, produce a path in the tree from the root down to the node that most closely ...
3
votes
2answers
102 views

How should the output layer of an LSTM be when the output are word embeddings?

I'm having trouble grasping how to output word embeddings from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot ...
4
votes
1answer
44 views

Do individual dimensions in vector space have meaning?

Word2vec assigns an N-dimensional vector to given words (which can be considered a form of dimensionality reduction). It turns out that, at least with a number of canonical examples, vector ...
-2
votes
1answer
104 views

I need a word database… Any qualities I should look for?

I need a word database to train from. I found a word2vec JS word vector database, but I need a method to teach it which words go in which patterns. Please note that I am not asking to have you ...
2
votes
2answers
58 views

Intuition on how word embeddings bring information to a network

How is it that word embedding layer (say word2vec) brings more insights to the network compared to a simple one hot encoded layer? I understand how word embedding carry some semantic meaning, but it ...
1
vote
1answer
67 views

How is the word embedding represented in the paper “Recurrent neural network based language model”?

I'm reading "Recurrent neural network based language model" of Mikolov et al. (2010). Although the article is straight forward, I'm not sure how word embedding $w(t)$ is obtained: The reason I wonder ...
-2
votes
1answer
42 views

Over-exposure of certain items in content based recommendation engine

I'm working on a content based recommendation engine for ebooks. I create document vectors with 300 features for every ebook using a word2vec model trained on google news and determine recommendations ...
2
votes
0answers
67 views

How can we build word embeddings for a language? [closed]

Which algorithms are there to create word embeddings for a given language?