New answers tagged

0 votes

Masking in Decoder of Transformer

No, adding an additional mask to the residual connections or the linear layer isn't necessary. The masking is crucial solely for the attention mechanism. Implementing a mask in the residual ...
Cesar Ruiz's user avatar
2 votes
Accepted

Why is it called multi-headed attention?

The original paper "Attention is all you need" mentions the following. " Instead of performing a single attention function with $ d_{model} $-dimensional keys, values and queries, we ...
Sathishkumar Thirumalai's user avatar
1 vote

Understanding the transformer at inference time

1 - yes, they are considered the conditioning of the model, so they are always known 2&3 - at inference, you start with a single <SOS> token, which will ...
Alberto's user avatar
  • 1,308
0 votes

While fine-tuning a decoder only LLM like LLaMA on chat dataset, what kind of padding should one use?

Decoder only text LLMs are autoregressive. Like any autoregressive function if you do not left pad there will be no prior token to your intial token for efficient gradient descent. In most text ...
Sam Marvasti's user avatar
0 votes

Why is dot-product and not Euclidean distance used for attention?

Dot-product attention and Euclidean distance-based attention are not necessarily similar, differing only in terms of computational cost. I wrote a recent paper to report that using the Euclidean ...
calmcc's user avatar
  • 101
0 votes

Why encoders are required in Transformers

The original encoder+decoder transformer (from attention is all you need) can perform different tasks compared to either the decoder-only transformer or the encoder-only transformer. The encoder+...
Robin van Hoorn's user avatar
0 votes

Is the Mask Needed for Masked Self-Attention During Inference with GPT-2

Yes, it is still needed. In fact, @mshlis has given a clear reason in the Answer to Q1. As I am new to the community and have no reputation to directly comment, let me supply his/her answer by opening ...
Zhifeng Jiang's user avatar

Top 50 recent answers are included