Questions tagged [decoder]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-1 votes
1 answer

Does a decoder in transformer model generate output embeddings like the following?

Encoder: ...
Steven's user avatar
  • 99
1 vote
1 answer

Masking in Decoder of Transformer

I understand that the masked multi-head attention block ensures that generation of token at time step t doesn't rely on subsequent tokens of the input. But the residual connection which adds the input ...
SAGALPREET SINGH's user avatar
3 votes
1 answer

Aren't context lengths for transformers an artificial restriction?

Let's focus on the case of decoder-only transformers, where I am using algorithm 10 from "Formal Algorithms for Transformers" by Mary Phung and Marcus Hutter as a reference. : https://i....
Robert Wegner's user avatar
0 votes
0 answers

What is the policy model in RLHF for LLMs?

What is the policy model doing explicitly in an LLM with RLHF setup? From my understanding, LLMs generate in a way that is no different from any of their predecessors: beam search decoding, ...
information_interchange's user avatar
0 votes
1 answer

Transformer decoder. Causal masking during inference?

I understand how causal masking in the self-attention layer of the decoder works and why we use it during training. What I want to ask is: should we use causal masking during inference ? Consider a ...
pi-tau's user avatar
  • 807