Questions tagged [embeddings]

For questions about embeddings (not necessarily just word embeddings, for which there is a specific tag) in the context of machine learning.

Filter by
Sorted by
Tagged with
1
vote
1answer
19 views

Perform clustering on high dimensional data

Recently I trained a BYOL model on a set of images to learn an embedding space where similar vectors are close by. The performance was fantastic when I performed approximate K-nearest neighbours ...
2
votes
1answer
75 views

What are knowledge graph embeddings?

What are knowledge graph embeddings? How are they useful? Are there any extensive reviews on the subject to know all the details? Note that I am asking this question just to give a quick overview of ...
2
votes
0answers
27 views

Is there any way to force one input have more effect on model?

Now I am working on building a deep learning model for a regression problem. I used 50 inputs and try to add one new categorical input. The problem is that this one input is much more important than ...
0
votes
1answer
25 views

Best way to resize 3d to 2d matrix

I have a (5, 128, 768) matrix, that is, I have 5 embedding spaces of shape (128, 768). Since they all keep a relation, and for ...
1
vote
0answers
19 views

How do the scale of an embedding affects a downstream task?

I am currently training a neural network in a self-supervised fashion, using Contrastive Loss and I want to use that network then to fine-tune it in a classification task with a small fraction of the ...
1
vote
0answers
14 views

What is the reason behind using node embeddings?

I was reading Chapter 3 from the following book (here) on graph representation learning. The chapter is about node embeddings. Question: What is the point of using node embeddings? Do we use them: to ...
0
votes
0answers
10 views

Mapping ground truth to downsampled embeddings

I am currently pulling embeddings out of the mid layers of PSPNet. I was wondering if anyone knows of a way to see what pixels in the ground truth map to the pixels in the intermediate layers? e.g. we ...
1
vote
0answers
18 views

Is graph embedding linear in its maintaining of graph geometry?

It is claimed that the main goal of graph embedding methods is to pack every node's properties into a vector with a smaller dimension, so node similarity in the original complex irregular spaces can ...
0
votes
0answers
18 views

Why use two different embeddings for actions in this paper?

I was reading this paper Top-𝐾 Off-Policy Correction for a REINFORCE Recommender System and I'm wondering is there a particular advantage to use different embeddings for actions, one embedding is ...
2
votes
2answers
216 views

How to determine the embedding size?

When we are training a neural network, we are going to determine the embedding size to convert the categorical (in NLP, for instance) or continuous (in computer vision or voice) information to hidden ...
0
votes
0answers
41 views

Anything similar to BERT but for pixel-wise embedding in images

In NLP there is BERT which can take a sentence and turn it into an embedding (vector representation) which in some ways encompasses the "meaning" or more precisely the context of the ...
1
vote
0answers
23 views

Why are BERT embeddings interpreted as representations of the corresponding words?

It's often assumed in literature that BERT embeddings are contextual representations of the corresponding word. That is, if the 5th word is "cold", then the 5th BERT embedding is a ...
2
votes
1answer
55 views

What is an "input embedding" in the context of NLP?

When reading about NLP, I saw it said that "input embeddings" are a main element of encoder-decoder learning frameworks for sequence modelling. What is an "input embedding" in the ...
5
votes
2answers
320 views

Is an embedding a representation of a word or its meaning?

What does the term "embedding" actually mean? An embedding is a vector, but is that vector a representation of a word or its meaning? Literature loosely uses the word for both purposes. ...
0
votes
0answers
26 views

Which well known node embedding algorithms to use for weighted graphs?

I am looking for a node representation learning algorithm to generate node embeddings that supports weighted graphs. I modified GCN to support weighted graphs, but I want to know an algorithm that ...
2
votes
2answers
114 views

Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?

In music information retrieval, one usually converts an audio signal into some kind "sequence of frequency-vectors", such as STFT or Mel-spectrogram. I'm wondering if it is a good idea to ...
0
votes
1answer
40 views

How does t-SNE preserves embedding orders?

According to the triplet loss Wikipedia page: t-SNE (t-distributed Stochastic Neighbor Embedding) preserves embedding orders via probability distributions, whereas triplet loss works directly on ...
2
votes
0answers
18 views

Embedding from Transformer-based model from paragraph or documnet (like Doc2Vec)

I have a set of data that contains the different lengths of sequences. On average the sequence length is 600. The dataset is like this: ...
0
votes
1answer
34 views

Are the Word2Vec encoded embeddings available online? [closed]

I am trying to do an NLP project and was wondering if there is anywhere online where the Word2Vec embeddings are stored (the actual n-dimmensional vectors). I want to search up a word and see what its ...
0
votes
1answer
33 views

Converting age and sex variables to a 64-unit dense layer

I am studying a preprint for my own learning (https://www.medrxiv.org/content/medrxiv/early/2020/04/27/2020.04.23.20067967.full.pdf) and I am befuddled by the following detail of the neural network ...
0
votes
0answers
39 views

Seq2Seq Modelling: when implementing some machine translation net, how are special tokens embedded?

When implementing any encoder-decoder network for machine translation, during training we provide the true output sentence to the decoder so that the context vector (from source language) may be ...
0
votes
0answers
42 views

How to change the number of input neurons in embedding layer?

I was building a recommender system using Tensorflow recommenders (TFRS) library . I was following the official tutorial for ranking model , where they have used two-tower model. The part where I have ...
0
votes
0answers
31 views

Sparse Multi-hot encoding and autoencoders

I'm working with graph neural networks. I have a large graph. Each node has 4 features [A,B,C,D]: 2 categorical with high cardinality: 86k (A) and 148k (B) different features 2 integer with ranges: [...
2
votes
1answer
63 views

How does the embeddings work in vision transformer from paper?

I get the part from the paper where the image is split into P say 16x16 (smaller images) patches and then you have to ...
1
vote
0answers
23 views

Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?

I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see Chiu and Nichols (2016) for the motivation). I haven't found any empirical evidence of whether or ...
2
votes
1answer
92 views

In the machine learning literature, what does it mean to say that something is "embedded" in some space?

In the machine learning literature, I often see it said that something is "embedded" in some space. For instance, that something is "embedded" in feature space, or that our data ...
1
vote
1answer
94 views

What is the role of embeddings in a deep recurrent Q network?

When describing the model architecture for a deep recurrent q network, the authors of the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning each agent consists of a recurrent ...
20
votes
5answers
8k views

What is the difference between latent and embedding spaces?

In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" ...
1
vote
1answer
2k views

How is the constraint $\|f(x)\|_{2}=1$ enforced for the embedding $f(x)$ in the FaceNet paper?

In the FaceNet paper, under section 3.2, the authors mention that: The embedding is represented by $f(x) \in \mathbb{R}^{d}$. It embeds an image $x$ into a $d$-dimensional Euclidean space. ...
3
votes
2answers
105 views

What is the intuition behind how word embeddings bring information to a neural network?

How is it that a word embedding layer (say word2vec) brings more insights to the neural network compared to a simple one-hot encoded layer? I understand how the word embedding carries some semantic ...