Skip to main content
Search type Search syntax
Tags [tag]
Exact "words here"
Author user:1234
user:me (yours)
Score score:3 (3+)
score:0 (none)
Answers answers:3 (3+)
answers:0 (none)
isaccepted:yes
hasaccepted:no
inquestion:1234
Views views:250
Code code:"if (foo != bar)"
Sections title:apples
body:"apples oranges"
URL url:"*.example.com"
Saves in:saves
Status closed:yes
duplicate:no
migrated:no
wiki:no
Types is:question
is:answer
Exclude -[tag]
-apples
For more details on advanced search visit our help page
Results tagged with
Search options not deleted user 26580

For questions about the concept of attention in artificial intelligence and machine learning. Attention-like mechanisms were successfully used in natural language processing and computer vision tasks, such as machine translation. For a review of attention-based mechanism used in NLP, take a look at "Attention in Natural Language Processing" by Andrea Galassi et al.

2 votes
1 answer
503 views

Is the Decoder mask (triangular mask) applied only in the first decoder block, or to all blo...

The Decoder mask, also called "look-ahead mask", is applied in the Decoder side to prevent it from attending future tokens. Something like this: [0, 1, 1, 1, 1] [0, 0, 1, 1, 1] [0, 0, 0, 1, 1] [0, 0, …
Leevo's user avatar
  • 305
6 votes
1 answer
234 views

Why are "Transformers" called this way?

What is the reason behind the name "Transformers", for Multi Head Self-Attention-based neural networks from Attention is All You Need? …
Leevo's user avatar
  • 305