I was reading the recent paper Graph Representation Learning via Hard and Channel-Wise Attention Networks, where the authors claim that there is no hard attention operator for graph data.

From my understanding, the difference between hard and soft attention is that for soft attention you're computing the attention scores between the nodes and all their neighbors while for hard attention you have a sampling function that selects only the most important neighbors. If that is the case, then GraphSage is an example of hard attention, because they apply the attention only on a subset of each node's neighbors.

Is my understanding of hard and soft attention wrong, or the claim that the authors made does not hold?


GraphSage does not have attention at all. Yes, it randomly samples (not most important as you claim) a subset of neighbors, but it does not compute attention score for each neighbor.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.