Hot answers tagged

13

The GPT-3 source code hasn’t been released but the creators say it uses the “same model and architecture as GPT-2” (source) with some exceptions. The GPT-2 source code is written in 100% Python. The model is based on Tensorflow and NumPy which are written using C and C++. My best guess is that GPT-3 is also written in Python using libraries based on C.


6

In sequence generating models, for vocabulary of size $N$ (number of words, parts of words, any other kind of token), one predicts the next token from distribution of the form: $$ \mathrm{softmax} (x_i/T) \quad i = 1, \ldots N, $$ Here $T$ is the temperature. The output of the softmax is the probability that the next token will be the $i$-th word in the ...


2

This is rather something for V&L Models, training on associated texts with circuit images. Data which should be hard to come by. I doubt these models are yet capable of catching enough detail in pictures for producing the desired results. I mean results that dont dissolve is smoke when soldering the circuit. Mapping from natural language to some formal ...


1

GPT-3 obviously knows this GPT-3 doesn't "know" things in the sense that it has learned specific translations, maths or science etc. GPT-3 is not a knowledge database or question answering system, but it can behave like one if prompted carefully. Sometimes it is hard to get specific responses. This appears to be one of those cases. Did I ask it ...


1

You may want to look at Chirpy Cardinal from Stanford. It doesn't provide mannerisms out of the box but the response generators can be configured to give each instance its own character. https://stanfordnlp.github.io/chirpycardinal/


Only top voted, non community-wiki answers of a minimum length are eligible