Linked Questions
13 questions linked to/from How does the (decoder-only) transformer architecture work?
97
votes
3
answers
87k
views
What is self-supervised learning in machine learning?
What is self-supervised learning in machine learning? How is it different from supervised learning?
- 41.4k
3
votes
1
answer
4k
views
Is it possible to use LLMs for regression tasks?
I want to use LLMs to predict edge weights in a graph based on attributes between two nodes. Is this even possible? If not, what would you recommend?
I tried to look up uses of LLM in regression tasks,...
2
votes
2
answers
2k
views
How does a LLM (transformer) pick words from its vocabulary?
I have a very rough understanding of the "attention/self attention" mechanism of transformer models and how this can be used to process a set of word vectors provided as an input/prompt to ...
- 21
3
votes
0
answers
2k
views
What is input (and shape) to K/V/Q of self-attention of EACH Decoder block of Language-translation model Transformer's tokens during Inference?
Transformer model of the original Attention paper has a decoder unit that works differently during Inference than Tranining.
I'm trying to understand the shapes used during decoder (both self-...
- 181
6
votes
2
answers
701
views
How does GPT-based language model like ChatGPT determine the n-th letter of a word?
I understand that GPT models process input text by converting words into tokens and then embedding vectors and do not process them letter by letter. Given this approach, I am curious to know how a ...
- 584
2
votes
1
answer
315
views
How can a transformer encoder attend to future tokens?
What does attending to future tokens mean? From my understanding, the transformer model works by inputting a prompt and predicting the next word in a sequence and this process just keeps repeating ...
- 21
0
votes
1
answer
946
views
How does Chat GPT encode a question?
Chat GPT is based on a decoder-only Transformer so it does not have an encoder. Given that, how is a user's question passed as input to Chat GPT's decoder? In a regular encoder-decoder architecture, ...
- 1
0
votes
1
answer
472
views
what are the applications scenarios for prefix decoder LMs
Motivated by this post wherein one of the comments mentioned the use-case for encoder-decoder LM. I wanted to know when to use prefix-decoder LM? vis a vis encoder-decoder or causal decoder only ...
- 1
2
votes
0
answers
288
views
How are weight matrices in attention learned?
I have been looking into transformers lately and have been reading tons of tutorials. All of them address the intuition behind attention, which I understand. However, they treat learning the weight ...
- 21
0
votes
1
answer
202
views
For a transformer decoder, how exactly are K, Q, and V for each decoding step?
For a transformer decoder, how exactly are K, Q, and V for each decoding step?
Assume my input prompt is "today is a" (good day).
At t= 0 (generation step 0):
K, Q, and V are the projections ...
- 183
0
votes
2
answers
517
views
How the Q,K,V be calculated in multi-head attention
I want to understand the transformer architecture, so I start with self attention and I understand their mechanism, but when I pass to the multi-head attention I find some difficulties like how ...
1
vote
1
answer
103
views
How are the parts of GPT connected?
Reading Stephen Wolfram's explanation of ChatGPT, it sounds as if first you train a very powerful "autocomplete" function that doesn't know anything specifically about chatbots, and then you ...
- 205