Questions tagged [embeddings]
For questions about embeddings (not necessarily just word embeddings, for which there is a specific tag) in the context of machine learning.
21
questions
0
votes
0answers
35 views
Anything similar to BERT but for pixel-wise embedding in images
In NLP there is BERT which can take a sentence and turn it into an embedding (vector representation) which in some ways encompasses the "meaning" or more precisely the context of the ...
1
vote
0answers
17 views
Why are BERT embeddings interpreted as representations of the corresponding words?
It's often assumed in literature that BERT embeddings are contextual representations of the corresponding word. That is, if the 5th word is "cold", then the 5th BERT embedding is a ...
1
vote
0answers
24 views
Do literature uses (text) encoding and (word) embedding in a loose sense?
Many research papers use the word "encoding" in many ways. One way is that they use the word "encoding" for models that do convert the text into a vector (say text encoding).
But, &...
1
vote
1answer
37 views
What is an “input embedding” in the context of NLP?
When reading about NLP, I saw it said that "input embeddings" are a main element of encoder-decoder learning frameworks for sequence modelling. What is an "input embedding" in the ...
5
votes
2answers
278 views
Is an embedding a representation of a word or its meaning?
What does the term "embedding" actually mean?
An embedding is a vector, but is that vector a representation of a word or its meaning? Literature loosely uses the word for both purposes. ...
0
votes
2answers
64 views
Book(s) for text embedding
Text here refers to either character or word or sentence.
Is there any recent textbook that encompasses from classical methods to the modern techniques for embedding texts?
If a single textbook is ...
0
votes
0answers
16 views
Which well known node embedding algorithms to use for weighted graphs?
I am looking for a node representation learning algorithm to generate node embeddings that supports weighted graphs.
I modified GCN to support weighted graphs, but I want to know an algorithm that ...
2
votes
1answer
52 views
Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?
In music information retrieval, one usually converts an audio signal into some kind "sequence of frequency-vectors", such as STFT or Mel-spectrogram.
I'm wondering if it is a good idea to ...
0
votes
1answer
29 views
How does t-SNE preserves embedding orders?
According to the triplet loss Wikipedia page:
t-SNE (t-distributed Stochastic Neighbor Embedding) preserves embedding orders via probability distributions, whereas triplet loss works directly on ...
2
votes
0answers
15 views
Embedding from Transformer-based model from paragraph or documnet (like Doc2Vec)
I have a set of data that contains the different lengths of sequences. On average the sequence length is 600. The dataset is like this:
...
0
votes
1answer
31 views
Are the Word2Vec encoded embeddings available online? [closed]
I am trying to do an NLP project and was wondering if there is anywhere online where the Word2Vec embeddings are stored (the actual n-dimmensional vectors).
I want to search up a word and see what its ...
0
votes
1answer
29 views
Converting age and sex variables to a 64-unit dense layer
I am studying a preprint for my own learning (https://www.medrxiv.org/content/medrxiv/early/2020/04/27/2020.04.23.20067967.full.pdf) and I am befuddled by the following detail of the neural network ...
0
votes
0answers
37 views
Seq2Seq Modelling: when implementing some machine translation net, how are special tokens embedded?
When implementing any encoder-decoder network for machine translation, during training we provide the true output sentence to the decoder so that the context vector (from source language) may be ...
0
votes
0answers
24 views
How to change the number of input neurons in embedding layer?
I was building a recommender system using Tensorflow recommenders (TFRS) library . I was following the official tutorial for ranking model , where they have used two-tower model. The part where I have ...
0
votes
0answers
19 views
Sparse Multi-hot encoding and autoencoders
I'm working with graph neural networks.
I have a large graph. Each node has 4 features [A,B,C,D]:
2 categorical with high cardinality: 86k (A) and 148k (B) different features
2 integer with ranges: [...
1
vote
0answers
26 views
How does the embeddings work in vision transformer from paper?
I get the part from the paper where the image is split into P say 16x16 (smaller images) patches and then you have to ...
1
vote
0answers
21 views
Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?
I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see Chiu and Nichols (2016) for the motivation). I haven't found any empirical evidence of whether or ...
2
votes
1answer
81 views
In the machine learning literature, what does it mean to say that something is “embedded” in some space?
In the machine learning literature, I often see it said that something is "embedded" in some space. For instance, that something is "embedded" in feature space, or that our data ...
1
vote
1answer
53 views
What is the role of embeddings in a deep recurrent Q network?
When describing the model architecture for a deep recurrent q network, the authors of the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning
each agent consists of a recurrent ...
19
votes
4answers
6k views
What is the difference between latent and embedding spaces?
In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" ...
2
votes
2answers
89 views
What is the intuition behind how word embeddings bring information to a neural network?
How is it that a word embedding layer (say word2vec) brings more insights to the neural network compared to a simple one-hot encoded layer?
I understand how the word embedding carries some semantic ...
