Questions tagged [decoder]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-1 votes
1 answer
179 views

Does a decoder in transformer model generate output embeddings like the following?

Encoder: ...
Steven's user avatar
  • 99
1 vote
1 answer
182 views

Masking in Decoder of Transformer

I understand that the masked multi-head attention block ensures that generation of token at time step t doesn't rely on subsequent tokens of the input. But the residual connection which adds the input ...
SAGALPREET SINGH's user avatar
3 votes
1 answer
566 views

Aren't context lengths for transformers an artificial restriction?

Let's focus on the case of decoder-only transformers, where I am using algorithm 10 from "Formal Algorithms for Transformers" by Mary Phung and Marcus Hutter as a reference. : https://i....
Robert Wegner's user avatar
0 votes
0 answers
21 views

What is the policy model in RLHF for LLMs?

What is the policy model doing explicitly in an LLM with RLHF setup? From my understanding, LLMs generate in a way that is no different from any of their predecessors: beam search decoding, ...
information_interchange's user avatar
0 votes
1 answer
776 views

Transformer decoder. Causal masking during inference?

I understand how causal masking in the self-attention layer of the decoder works and why we use it during training. What I want to ask is: should we use causal masking during inference ? Consider a ...
pi-tau's user avatar
  • 807