I read this In Google AI blog: link: https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html

S1 : " the Transformer can learn to immediately attend to the word “river” and make this decision in a single step."

Q1: What is the criteria applied for 'immediate attention' ? Q2: What is the logic used to select a word for 'immediate attention'? Q3: What is single step? After which step is this single step?

S2: " the disambiguating “river” could receive a high attention score "

Q4. How does one determine a word is 'disambiguating' or not?

S3: " to compute the next representation for a given word - “bank” for example - the Transformer compares it to every other word in the sentence."

Q5. Is this sentence contradictory to the first sentence quoted here - see S1 above

Please, help me to understand the mechanism better.

  • 2
    $\begingroup$ Hi Krishna! Please, ask only one question per post. If you have multiple questions, create a post for each of them. It is unlikely that future readers of this post or visitors of this website will have the same combination of questions, so this makes your post with multiple questions less useful. $\endgroup$ – nbro Oct 3 '19 at 1:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.