As I understand it, GPT-style LLMs take a sequence of tokens as input and output a token probability vector. The first thing that happens to an input token is that it goes through the input embedding, which can be thought of multiplying the embedding matrix with a one-hot token vector.

If that is correct, nothing is stopping us from using a probability vector in place of a one-hot vector as input to the LLM.

Question 1: What happens when the input to the LLM contains probability vectors instead of discrete tokens? For example if the token "dog" was [0,0,1,0,0,0,0] and the token "cat" was [0,0,0,1,0,0,0], what if the input instead contained the pseudo-token [0,0,0.5,0.5,0,0,0]?

Question 2: When generating a text, we turn the output probability distribution into a single token by sampling and append the new token. What if we don't sample and instead append the probability vector itself? Of course, this would not produce human-readable text.

When using Chain-of-Thought prompting, we want the model to slowly reason through the problem, but could it not be helpful to let it output full probability vectors at this stage instead of forcing it to discretize into human-readable tokens?

Or would none of this produce meaningful results because models are only trained on discrete tokens?

  • $\begingroup$ LLM may not greedily chose only the next token with highest probability (depending on temperature), but use a kind of beam search to look at further tokens as well and only keep the most promising beams. However this is much more costly. $\endgroup$
    – Lelouch
    Commented Apr 5 at 8:35

1 Answer 1

  1. it samples from such distribution, thus would wither sample the one-hot for dog or for cat
  2. then what's the point of using it if it's not human interpretable?
  3. no, we feed the sampled token to the transformer because it cannot know wether we sampled (in you example) cat or dog. Feeding back the whole distribution would give him no more information than not giving it none

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .