I've found this article that seems to answer my question:
From this, my understanding is that inference-time describes when a machine learning system is put into use following training; so basically at the time of task application.
I think this would mean that the paper's authors are stating that the ...
Positional Encodings in Transformers exist to give the model some information about the position of the embedding. This makes sense in fields like NLP or Time Series Data, since the position(order) matters in this case.
However, since you say that order of the data is not relevant in your use case, positional encoding would not be necessary.
Language models produce a probability distribution over a set of words. You determine the next word by sampling from this distribution. So, determining the next word is stochastic even though the distribution is the same given the initial prompt.
OP seems to think value, query and keys are supposed to be different in the original Vaswani multi-head attention. As can be seen in Keras' documentation on their implementation of the multi-headed attention layer, "If query, key, value are the same, then this is self-attention."
One thing missing from the graphics you use are the ...
Yes, they can handle sequences with arbitrary length sequence, but with some remarks.
In the paper Training data-efficient image transformers & distillation through attention authors train models in the resolution 224x224 (1 + 14x14 tokens) and then finetune to the 384x384 (1 + 28x28 tokens).
Weights to produce queries, keys, values, as well as ...
Attention mechanism solves this problem by allowing the decoder to “look-back” at the encoder's hidden states based on its current state. This allows the decoder to extract only relevant information about the input tokens at each decoding, thus learning more complicated dependencies between the input and the output.