Search Results
| Search type | Search syntax |
|---|---|
| Tags | [tag] |
| Exact | "words here" |
| Author |
user:1234 user:me (yours) |
| Score |
score:3 (3+) score:0 (none) |
| Answers |
answers:3 (3+) answers:0 (none) isaccepted:yes hasaccepted:no inquestion:1234 |
| Views | views:250 |
| Code | code:"if (foo != bar)" |
| Sections |
title:apples body:"apples oranges" |
| URL | url:"*.example.com" |
| Saves | in:saves |
| Status |
closed:yes duplicate:no migrated:no wiki:no |
| Types |
is:question is:answer |
| Exclude |
-[tag] -apples |
| For more details on advanced search visit our help page | |
Results tagged with transformer
Search options answers only
not deleted
user 30426
For questions related to the transformer, which is a deep machine learning model introduced in 2017 in the paper "Attention Is All You Need", used primarily in the field of natural language processing (NLP).
2
votes
Why do the authors of the T5 paper say that the "architectural changes are orthogonal to the...
Looking at the paper, it seems to me that they are not using orthogonal in a literal, mathematics (or geometric) sense. Instead, I read that as two things (especially since the word "ablation" appear …
- 951
8
votes
How can Transformers handle arbitrary length input?
We have found it useful to wrap our transformer in a class that allows us to programmatically use a sliding window across inputs that are longer than the supported transformer input length. … If it is longer, we iteratively slide across the data, passing each window into the transformer and then aggregate the outputs. …
- 951
2
votes
From where do the Encoders in Transformers gets Input Embedding from?
Either by building embeddings yourself or loading pretrained embeddings.
For building yourself, this is typically done with an auto-regressive model. It can be as simple as creating numeric represent …
- 951
0
votes
Why does this multiplication of $Q$ and $K$ have a variance of $d_k$, in scaled dot product ...
It might help to take two small matrices that match the assumptions (mean of zero and variance of one) and just do the matrix multiplication. The dimensionality of K scales Q in the multiplication, s …
- 951