The seminal Attention is all you need paper introduces Transformers and implements the attention mecanism with "queries, keys, values", in an analogy to a retrieval system.
I understand the whole process of multi-head attention and such (i.e., what is done with the Q, K, V values and why), but I'm confused on how these values are computed in the first place. AFAICT, the paper seems to completely leave that out.
Both Figure 2 of the paper and equations explaining Attention and Multihead attention start with Q,K,V already there :
The answers regaridng the origin of Q,K,V I've found so far haven't satisfied me :
In this similar question, the accepted answer says "The proposed multihead attention alone doesn't say much about how the queries, keys, and values are obtained, they can come from different sources depending on the application scenario.".
I also see some answers (eg this one on the same question) which say that Q, K and V are the result of multiplication of the input embedding with some matrices. This is also what is shown in the popular blog post The Illustrated Transformer :
Why isn't the computing of Q,K,V -be it "left to the application" or "multiplication with matrices" made more clear in the paper, at the very least for the task of language translation for which they show some results and so obviously did compute Q,K,V in some way ? If it is matrix multiplication, are these matrices ($W^Q$, etc in the figure of the blog post) trained with backprop jointly with the rest of the network or pretrained ? What are the resulting shapes of Q,K,V ?