So in Graph Neural Network (GNN) we have node embeddings which is a feature vector that describes the node, is it analogous to hidden layer of Artificial neural network such as feed-forward neural network? After all hidden layer stores set of weights and biases to extract some "feature", which begs the question if node embeddings are analogous to a hidden layer?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ In my understanding embedding is a static vector, but hidden layer is a function. But this is a good question as hidden layer is indeed extracting a feature, while node embedding represents features of the node. $\endgroup$– gfdsalMar 5, 2022 at 21:21
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
Embeddings are vectors. Layers are functions.
So, node embeddings (e.g. produced by TransE) are analogous to word embeddings or code embeddings, i.e. they are vector (and lower-dimensional) representations of the original objects, such that similar objects have similar embeddings, for some notion of similarity.
Your question is analogous to "Are convolutional layers analogous to feature maps?". So, you could use neural networks with layers to learn embeddings, but this does not have to be the case. For example, TransE does not "really" use a neural network (i.e. a sequence of layers that compute a non-linear function of the inputs), but it just optimizes the $L_2$ norm of a simple function of the vector representation of the nodes and edges, although it uses gradient descent.
-
$\begingroup$ makes so much sense, so that means embeddings are linked to model they are learned with? I thought embeddings are more universal as a space that separate two dissimilar things but here it appears that to use the embedding we need, the network on which it has been learned on, as well $\endgroup$– user0193Mar 30, 2022 at 6:12