New answers tagged gpt-3
5
In sequence generating models, for vocabulary of size $N$ (number of words, parts of words, any other kind of token), one predicts the next token from distribution of the form:
$$
\mathrm{softmax} (x_i/T) \quad i = 1, \ldots N,
$$
Here $T$ is the temperature. The output of the softmax is the probability that the next token will be the $i$-th word in the ...
Top 50 recent answers are included