2
$\begingroup$

From my understanding, GPT-3 is trained on predicting the next token from a sequence of tokens. Given this, how is it able to take commands? For instance, in this example input, wouldn't the statistically most likely prediction be to insert a period and end the sentence?

Input: write me a beautiful sentence

Output: I cannot put into words how much I love you, so I'll just say it's infinite.
$\endgroup$

1 Answer 1

0
$\begingroup$

No, it's actually a very precise guess determined by relatively complex statistical equations.

GPT-3 is able to take commands because it has been trained on a dataset of commands. This means that it is able to recognize patterns in sequences of tokens that indicate a command. In the example input, the most likely prediction is to insert a period and end the sentence because this is what typically happens at the end of a sentence. However, GPT-3 is able to take commands because it is able to recognize patterns that indicate a different kind of sequence, such as a command.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .