1
vote
$\begingroup$

I know it cost around $4.3 million dollars to train, but how much computing power does it cost to run the finished program? IBM Watson chatbot AI only costs a few cents per chat message to use, OpeenAI Five seemed to run on a single gaming PC setup. So I'm wondering how much computing power does it need to run the finished ai program.

$\endgroup$
2

2 Answers 2

2
votes
$\begingroup$

I can't anwser your question on how much computing power you might need, but you'll need atleast a smallgrid to run the biggest model just looking at the memory requirments (175B parameters so 700GB of memory). The biggest gpu has 48 GB of vram
I've read that gtp-3 will come in eigth sizes, 125M to 175B parameters. So depending upon which one you run you'll need more or less computing power and memory.
(https://lambdalabs.com/blog/demystifying-gpt-3/)
For an idea of the size of the smallest, "The smallest GPT-3 model is roughly the size of BERT-Base and RoBERTa-Base."

$\endgroup$
1
vote
$\begingroup$

I think it is premature to answer your question as OpenAI has not made GPT-3 available yet other than via a web-based API. For more information see OpenAI API.

From OpenAI will start selling its text-generation tech, and the first customers include Reddit, by James Vincent:

Access to the GPT-3 API is invitation-only, and pricing is undecided.

You can join the OpenAI wait list here: https://beta.openai.com/

I read somewhere that to load GPT-3 for inferencing requires 300GB if using half-precision floating point (FP16). There are no GPU cards today that even in a set of four will provide 300GB of video RAM. For example, the best I believe you can do in a single desktop box is four NVLinked Nvidia RTX 8000 cards on a single motherboard. Each card has 48GB of VRAM each. That would only provide a total of 192GB of VRAM.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged .