I was reading the recent paper Graph Representation Learning via Hard and Channel-Wise Attention Networks, where the authors claim that there is no hard attention operator for graph data.

From my understanding, the difference between hard and soft attention is that for soft attention you're computing the attention scores between the nodes and all their neighbors while for hard attention you have a sampling function that selects only the most important neighbors. If that is the case, then GraphSage is an example of hard attention, because they apply the attention only on a subset of each node's neighbors.

Is my understanding of hard and soft attention wrong, or the claim that the authors made does not hold?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.