0
$\begingroup$

I am doing a machine translation task using a Graph2Seq graph neutral network.

There are many different variants of GNN:

  • GCN
  • GAT
  • GraphSage
  • GGNN

Which one would be the most effective for a machine translation task?

I have around 400 nodes in the graph, and on average, a node is connected to 3.5 neighbours.

$\endgroup$
2
  • $\begingroup$ The choice of GNN depends highly on the nature of the graphs you are dealing with, so the information you are giving is not enough to argue about which architecture might be best: How do you construct the graphs? How large are the graphs and what is the size of the node neighborhoods are some questions to consider. Even then you can only assume which GNN might work, you will still have to do empirical evaluations on different GNNs to be sure $\endgroup$
    – Chillston
    Mar 31 at 12:04
  • $\begingroup$ @Chillston added information about the graph $\endgroup$
    – Exploring
    Mar 31 at 18:01

1 Answer 1

1
$\begingroup$

I will start by saying that I do not have any experience with Graph2Seq networks or GGNN, but I have some knowledge about GNN in general and the other three architectures.

Firstly, it is essential to define your definition of effectiveness because each of these architectures excels at different things. For instance, GraphSage is most of the time the least expressive of these networks but will scale best to larger graphs, something that GGNN and GAT can not do as easily on most computers.

In most cases, it is best to look at previous literature to see which architecture they used for their implementation. Lastly, it is good to experiment with each architecture. For example, you could start with GCN and see how the performance is. If the GCN is accurate enough, stay with it or try GraphSage, which takes less computation and is thus, in most cases, faster. If GCN is not accurate enough, I recommend trying either GAT or GGNN, which should be more expressive.

Explanation Expressivity

In the context of GNN, expressivity means how much relevant information and patterns the GNN architecture can capture from the input graph. GraphSage has the worst expressivity because it does not use all the node neighbours when aggregating information, but only samples some of the node neighbours. Sampling is used because the belief is that most nodes in a neighbourhood share similar node features. However, this does not have to be the case and could mean that GraphSage will miss important information.

GAT is the opposite of this. GAT uses the information of all its neighbours in the same way as a GCN does and weighs the importance of those features with an attention mechanism. These two aspects make it more expressive because it will use all the available information and weigh the importance of neighbouring node features. Important information will be captured more easily by GAT for this reason.

Based on your graph

Your graph is quite small and should work fine with either GAT, GGNN or GCN. GraphSage is designed for significantly larger graphs and does not work with edge attributes. Again, I advise you to look at previous literature about Graph2Seq, to see what kind of architecture they use. Also, add more information about the graph like if it is directed or undirected, and information about if it only has node features or also edge weighs and attributes.

$\endgroup$
7
  • $\begingroup$ would you mind you explain the expressiveness of GCN, GAT and GGNN in a bit more detail. Would really appreciate it. $\endgroup$
    – Exploring
    Mar 31 at 18:05
  • $\begingroup$ would you mind to explain why some of the GNN variants are not expressive enough? $\endgroup$
    – Exploring
    Apr 1 at 8:34
  • 1
    $\begingroup$ I edited my initial answer to include more information about expressiveness. $\endgroup$
    – Jesse
    Apr 1 at 13:25
  • $\begingroup$ I have been struggling firstly where to ask this question. Secondly finding materials about this topic. Appreciate it. $\endgroup$
    – Exploring
    Apr 1 at 17:36
  • $\begingroup$ I am using a directed graph. $\endgroup$
    – Exploring
    Apr 5 at 18:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .