New answers tagged transformer
0
votes
Masking in Decoder of Transformer
No, adding an additional mask to the residual connections or the linear layer isn't necessary. The masking is crucial solely for the attention mechanism. Implementing a mask in the residual ...
- 453
2
votes
Accepted
Why is it called multi-headed attention?
The original paper "Attention is all you need" mentions the following.
"
Instead of performing a single attention function with $ d_{model} $-dimensional keys, values and queries,
we ...
1
vote
Understanding the transformer at inference time
1 - yes, they are considered the conditioning of the model, so they are always known
2&3 - at inference, you start with a single <SOS> token, which will ...
- 1,308
0
votes
While fine-tuning a decoder only LLM like LLaMA on chat dataset, what kind of padding should one use?
Decoder only text LLMs are autoregressive. Like any autoregressive function if you do not left pad there will be no prior token to your intial token for efficient gradient descent.
In most text ...
0
votes
Why is dot-product and not Euclidean distance used for attention?
Dot-product attention and Euclidean distance-based attention are not necessarily similar, differing only in terms of computational cost. I wrote a recent paper to report that using the Euclidean ...
- 101
0
votes
Why encoders are required in Transformers
The original encoder+decoder transformer (from attention is all you need) can perform different tasks compared to either the decoder-only transformer or the encoder-only transformer.
The encoder+...
- 2,083
0
votes
Is the Mask Needed for Masked Self-Attention During Inference with GPT-2
Yes, it is still needed. In fact, @mshlis has given a clear reason in the Answer to Q1. As I am new to the community and have no reputation to directly comment, let me supply his/her answer by opening ...
Top 50 recent answers are included