New answers tagged

5

In sequence generating models, for vocabulary of size $N$ (number of words, parts of words, any other kind of token), one predicts the next token from distribution of the form: $$ \mathrm{softmax} (x_i/T) \quad i = 1, \ldots N, $$ Here $T$ is the temperature. The output of the softmax is the probability that the next token will be the $i$-th word in the ...


Top 50 recent answers are included