Skip to main content
2 votes
Accepted

Why is there a shared matrix W in graph attention networks instead of the query-key-value trio like in regular transformers?

To my understanding, there isn't any theoretical reason why the query, key and values weights are absent. I feel that the difference may lie in the way the additive attention is calculated vs the dot-...
Cesar Ruiz's user avatar
2 votes

How to pass batch of Graphs in GATv2Conv?

For efficiency reasons, it is the standard for graph learning libraries to use the format [n_batches x num_nodes, node_feature_dim], instead of having batches in an ...
Chillston's user avatar
  • 1,748
2 votes

How can I improve this toy Graph Neural Network Generative Language model

First of all, I would like to encourage you to keep trying new things, it sounds super fun! There are a few things I would like you to clarify about Graph Neural Networks (GNNs) and Graph ...
Cesar Ruiz's user avatar
1 vote
Accepted

sample one node from each graph in a batch with pytorch

This question is better suited for StackOverflow has it is more about coding rather than AI per se, but I did something like this recently, so here is what I ended up with: First, you generate a ...
Chillston's user avatar
  • 1,748
1 vote

How does Training and Validating work with Graph CNNs

There are two main settings of doing training/validation/testing in graphs: Inductive Setting: You datasets consists of multiple graph (e.g. molecules) and you simply split the graphs into training/...
Chillston's user avatar
  • 1,748
1 vote

How to do image classification with optional metadata?

Train a self-supervised model to generate embeddings of your metadata Train a model to generate embeddings of your images (or use/fine-tune a pretrained image model) Train a prediction model that ...
Karl's user avatar
  • 206
1 vote
Accepted

How to do image classification with optional metadata?

You could use your existing CNN architecture and concatenate the metadata with the flattened last convolutional layer. Add a flag Boolean feature for whether the metadata is present, and if it is not, ...
Neil Slater's user avatar
  • 32.7k
1 vote

What is GNN Cheatsheet in PyG Docs

Regarding "what the purpose of this is as it shows a table and there are tick marks if checked.", it sounds like that the tick marks indicating convolution operation support. This means you ...
Cloud Cho's user avatar
  • 181

Only top scored, non community-wiki answers of a minimum length are eligible