Skip to main content
edited title
Link
nbro
  • 41.4k
  • 12
  • 114
  • 205

Quick question on autoregressive LLM For a transformer decoder, how exactly are K, Q, and V for each decoding step?

added 18 characters in body
Source Link
Robin van Hoorn
  • 2.7k
  • 2
  • 11
  • 33

For LLMa transformer decoder, how exactly is theare K, Q, and V for each decoding step?

SayAssume my input prompt is "today is a" (good day).

At t= 0 (generation step 0): K, Q, and V are the projections of the sequence ("today is a") Then say the next token generated is "good"

At t= 1(generation step 1): Which one is true:.

  • K, Q, V are the projections of the sequence ("today is a good")

ORAt t=1 (generation step 1): Which one is true:

  • K, Q, are the projections of the sequence ("today is a") , V is the projection of sequence ("good")?
  1. K, Q, and V are the projections of the sequence ("today is a good")
  2. K, Q, are the projections of the sequence ("today is a"), and V is the projection of the sequence ("good")?

For LLM decoder, how exactly is the K, Q, V for each decoding step?

Say my input prompt is "today is a" (good day).

At t= 0 (generation step 0): K, Q, V are the projections of the sequence ("today is a") Then say the next token generated is "good"

At t= 1(generation step 1): Which one is true:

  • K, Q, V are the projections of the sequence ("today is a good")

OR

  • K, Q, are the projections of the sequence ("today is a") , V is the projection of sequence ("good")?

For a transformer decoder, how exactly are K, Q, and V for each decoding step?

Assume my input prompt is "today is a" (good day).

At t= 0 (generation step 0): K, Q, and V are the projections of the sequence ("today is a") Then say the next token generated is "good".

At t=1 (generation step 1): Which one is true:

  1. K, Q, and V are the projections of the sequence ("today is a good")
  2. K, Q, are the projections of the sequence ("today is a"), and V is the projection of the sequence ("good")?
Source Link
wrek
  • 183
  • 4

Quick question on autoregressive LLM?

For LLM decoder, how exactly is the K, Q, V for each decoding step?

Say my input prompt is "today is a" (good day).

At t= 0 (generation step 0): K, Q, V are the projections of the sequence ("today is a") Then say the next token generated is "good"

At t= 1(generation step 1): Which one is true:

  • K, Q, V are the projections of the sequence ("today is a good")

OR

  • K, Q, are the projections of the sequence ("today is a") , V is the projection of sequence ("good")?