Search type Search syntax
Tags [tag]
Exact "words here"
Author user:1234
user:me (yours)
Score score:3 (3+)
score:0 (none)
Answers answers:3 (3+)
answers:0 (none)
isaccepted:yes
hasaccepted:no
inquestion:1234
Views views:250
Code code:"if (foo != bar)"
Sections title:apples
body:"apples oranges"
URL url:"*.example.com"
Bookmarks inbookmarks:mine
inbookmarks:1234
Status closed:yes
duplicate:no
migrated:no
wiki:no
Types is:question
is:answer
Exclude -[tag]
-apples
For more details on advanced search visit our help page
Results tagged with
Search options user 43632

For questions related to the transformer, which is a deep machine learning model introduced in 2017 in the paper "Attention Is All You Need", used primarily in the field of natural language processing (NLP).

5
votes
3answers
In the tutorial linked above, the transformer is implemented from scratch and nn.Embedding from pytorch is used for the embeddings. …
asked Feb 5 '21 by Bert Gayus
3
votes
I have found a good answer in this blog post The Transformer: Attention Is All You Need: we learn a “word embedding” which is a smaller real-valued vector representation of the word that carries some … The Transformer uses a random initialization of the weight matrix and refines these weights during training – i.e. it learns its own word embeddings. …
answered Feb 6 '21 by Bert Gayus
1
vote
0answers
Update: In the tutorial The Illustrated GPT-2 (Visualizing Transformer Language Models) I found an explanation for GPT-2 which seems to be similar to my question. …
asked Feb 8 '21 by Bert Gayus
0
votes
2answers
Since pose estimation is often a task where spatial-temporal context should be helpful in finding subsequent key points, I thought there should be many papers on it. However, I could not find any work …
asked Jan 11 by Bert Gayus