I was reading the recent paper Graph Representation Learning via Hard and Channel-Wise Attention Networks, where the authors claim that there is no hard attention operator for graph data.

From my understanding, the difference between hard and soft attention is that for soft attention you're computing the attention scores between the nodes and all their neighbors while for hard attention you have a sampling function that selects only the most important neighbors. If that is the case, then GraphSage is an example of hard attention, because they apply the attention only on a subset of each node's neighbors.

Is my understanding of hard and soft attention wrong, or the claim that the authors made does not hold?

  • $\begingroup$ If you now know the answer to this question, I suggest that you review the answer below and accept it if it answers the question. If not, feel free to leave an answer to your own question and accept it. Then please flag this comment as "no longer needed". $\endgroup$ – nbro Jan 6 at 23:40

GraphSage does not have attention at all. Yes, it randomly samples (not most important as you claim) a subset of neighbors, but it does not compute attention score for each neighbor.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.