Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
I am doing a machine translation task using a Graph2Seq graph neutral network.
There are many different variants of GNN:
Which one would be the most effective for a machine translation task?
I have around 400 nodes in the graph, and on average, a node is connected to 3.5 neighbours.
I will start by saying that I do not have any experience with Graph2Seq networks or GGNN, but I have some knowledge about GNN in general and the other three architectures.
Firstly, it is essential to define your definition of effectiveness because each of these architectures excels at different things. For instance, GraphSage is most of the time the least expressive of these networks but will scale best to larger graphs, something that GGNN and GAT can not do as easily on most computers.
In most cases, it is best to look at previous literature to see which architecture they used for their implementation. Lastly, it is good to experiment with each architecture. For example, you could start with GCN and see how the performance is. If the GCN is accurate enough, stay with it or try GraphSage, which takes less computation and is thus, in most cases, faster. If GCN is not accurate enough, I recommend trying either GAT or GGNN, which should be more expressive.
In the context of GNN, expressivity means how much relevant information and patterns the GNN architecture can capture from the input graph. GraphSage has the worst expressivity because it does not use all the node neighbours when aggregating information, but only samples some of the node neighbours. Sampling is used because the belief is that most nodes in a neighbourhood share similar node features. However, this does not have to be the case and could mean that GraphSage will miss important information.
GAT is the opposite of this. GAT uses the information of all its neighbours in the same way as a GCN does and weighs the importance of those features with an attention mechanism. These two aspects make it more expressive because it will use all the available information and weigh the importance of neighbouring node features. Important information will be captured more easily by GAT for this reason.
Your graph is quite small and should work fine with either GAT, GGNN or GCN. GraphSage is designed for significantly larger graphs and does not work with edge attributes. Again, I advise you to look at previous literature about Graph2Seq, to see what kind of architecture they use. Also, add more information about the graph like if it is directed or undirected, and information about if it only has node features or also edge weighs and attributes.
