Questions tagged [embeddings]
For questions about embeddings (not necessarily just word embeddings, for which there is a specific tag) in the context of machine learning.
24
questions
1
vote
0answers
10 views
What is the reason behind using node embeddings?
I was reading Chapter 3 from the following book (here) on graph representation learning. The chapter is about node embeddings.
Question: What is the point of using node embeddings?
Do we use them:
to ...
0
votes
0answers
10 views
Mapping ground truth to downsampled embeddings
I am currently pulling embeddings out of the mid layers of PSPNet. I was wondering if anyone knows of a way to see what pixels in the ground truth map to the pixels in the intermediate layers? e.g. we ...
0
votes
0answers
17 views
Is graph embedding linear in its maintaining of graph geometry?
It is claimed that the main goal of graph embedding methods is to pack every node's properties into a vector with a smaller dimension, so node similarity in the original complex irregular spaces can ...
0
votes
0answers
17 views
Why use two different embeddings for actions in this paper?
I was reading this paper Top-𝐾 Off-Policy Correction for a REINFORCE Recommender System and I'm wondering is there a particular advantage to use different embeddings for actions, one embedding is ...
2
votes
2answers
77 views
How to determine the embedding size?
When we are training a neural network, we are going to determine the embedding size to convert the categorical (in NLP, for instance) or continuous (in computer vision or voice) information to hidden ...
0
votes
0answers
36 views
Anything similar to BERT but for pixel-wise embedding in images
In NLP there is BERT which can take a sentence and turn it into an embedding (vector representation) which in some ways encompasses the "meaning" or more precisely the context of the ...
1
vote
0answers
21 views
Why are BERT embeddings interpreted as representations of the corresponding words?
It's often assumed in literature that BERT embeddings are contextual representations of the corresponding word. That is, if the 5th word is "cold", then the 5th BERT embedding is a ...
1
vote
1answer
41 views
What is an "input embedding" in the context of NLP?
When reading about NLP, I saw it said that "input embeddings" are a main element of encoder-decoder learning frameworks for sequence modelling. What is an "input embedding" in the ...
5
votes
2answers
309 views
Is an embedding a representation of a word or its meaning?
What does the term "embedding" actually mean?
An embedding is a vector, but is that vector a representation of a word or its meaning? Literature loosely uses the word for both purposes. ...
0
votes
0answers
21 views
Which well known node embedding algorithms to use for weighted graphs?
I am looking for a node representation learning algorithm to generate node embeddings that supports weighted graphs.
I modified GCN to support weighted graphs, but I want to know an algorithm that ...
2
votes
2answers
86 views
Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?
In music information retrieval, one usually converts an audio signal into some kind "sequence of frequency-vectors", such as STFT or Mel-spectrogram.
I'm wondering if it is a good idea to ...
0
votes
1answer
31 views
How does t-SNE preserves embedding orders?
According to the triplet loss Wikipedia page:
t-SNE (t-distributed Stochastic Neighbor Embedding) preserves embedding orders via probability distributions, whereas triplet loss works directly on ...
2
votes
0answers
15 views
Embedding from Transformer-based model from paragraph or documnet (like Doc2Vec)
I have a set of data that contains the different lengths of sequences. On average the sequence length is 600. The dataset is like this:
...
0
votes
1answer
32 views
Are the Word2Vec encoded embeddings available online? [closed]
I am trying to do an NLP project and was wondering if there is anywhere online where the Word2Vec embeddings are stored (the actual n-dimmensional vectors).
I want to search up a word and see what its ...
0
votes
1answer
30 views
Converting age and sex variables to a 64-unit dense layer
I am studying a preprint for my own learning (https://www.medrxiv.org/content/medrxiv/early/2020/04/27/2020.04.23.20067967.full.pdf) and I am befuddled by the following detail of the neural network ...
0
votes
0answers
37 views
Seq2Seq Modelling: when implementing some machine translation net, how are special tokens embedded?
When implementing any encoder-decoder network for machine translation, during training we provide the true output sentence to the decoder so that the context vector (from source language) may be ...
0
votes
0answers
33 views
How to change the number of input neurons in embedding layer?
I was building a recommender system using Tensorflow recommenders (TFRS) library . I was following the official tutorial for ranking model , where they have used two-tower model. The part where I have ...
0
votes
0answers
22 views
Sparse Multi-hot encoding and autoencoders
I'm working with graph neural networks.
I have a large graph. Each node has 4 features [A,B,C,D]:
2 categorical with high cardinality: 86k (A) and 148k (B) different features
2 integer with ranges: [...
1
vote
0answers
34 views
How does the embeddings work in vision transformer from paper?
I get the part from the paper where the image is split into P say 16x16 (smaller images) patches and then you have to ...
1
vote
0answers
22 views
Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?
I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see Chiu and Nichols (2016) for the motivation). I haven't found any empirical evidence of whether or ...
2
votes
1answer
82 views
In the machine learning literature, what does it mean to say that something is "embedded" in some space?
In the machine learning literature, I often see it said that something is "embedded" in some space. For instance, that something is "embedded" in feature space, or that our data ...
1
vote
1answer
65 views
What is the role of embeddings in a deep recurrent Q network?
When describing the model architecture for a deep recurrent q network, the authors of the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning
each agent consists of a recurrent ...
19
votes
5answers
7k views
What is the difference between latent and embedding spaces?
In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" ...
2
votes
2answers
91 views
What is the intuition behind how word embeddings bring information to a neural network?
How is it that a word embedding layer (say word2vec) brings more insights to the neural network compared to a simple one-hot encoded layer?
I understand how the word embedding carries some semantic ...
