Ravanbakhsh has clearly stated the relation between equivariance and parameter-sharing in neural networks. What I'm missing though, is where (and how) this relation becomes clear by considering the classic message passing layer of graph neural nets. In particular, the following example is provided in the above paper:

Let $A \in \{0,1\}^{N \times N}$ the adjacency matrix corresponding to a graph $\Lambda$. If we consider $\textbf{x} \in \mathbb{R}^N$ a node-feature vector, to process it in a neural net with parameters matrix $W$, we have to consider $W = w_1A\textbf{x} + w_2\textbf{x}$ in order to respect the symmetry of the graph $\Lambda$.

Now, looking at how the message-passing layer is defined in the Hamilton book, we have:

enter image description here

which represents the k-th update for the node $u$. Now, I'm not sure if the structure of $W_{self}$ and $W_{neigh}$ here satisfies the one illustrated by Ravanbakhsh, i.e. do these matrices have all the non-diagonal entries tided together, as well as the diagonal ones? I'm pretty confused to be honest. I think that my confusion might arise also from wrongly considering equivariance with respect to how the features are stacked together, rather then considering how the nodes in the graph can be permuted.. I really hope someone might provide clarity here.



You must log in to answer this question.

Browse other questions tagged .