So I asked ChatGPT-4 about its energy use per query. The dialog is appended at the bottom of this message. (I realize the OP asked about ChatGPT-3)
TL;DR
If ChatGPT-4's claim that a typical answer requires 30 TFLOPs on modern AI hardware is true then the computational energy per answer is typically 300 watt-seconds (less than 0.1 kWh), ignoring communication and other overhead.
Discussion
ChatGPT-4 stated that most queries are answered with a single forward pass through the model requiring about 30 trillion FLOPs. It produced a calculation this requires around 0.000833 kWh but the calculation contains at least two errors:
- It uses FLOPs/w to compute watt-seconds.
- It screws up the conversion of watt-second to
kWh by a factor of 1000.
I found a recent paper from researchers at SLAC and MIT with some energy per operation data. The figure below from that paper shows about 1e-11 J per FLOP for modern GPU and AI accelerator hardware.
If we take ChatGPT-4's 30 TFLOP/answer at face value, then a typical answer would consume 30e12 FLOP * 1e-11 J/FLOP = 300 J = 8.33e-5 kWH
This is a factor of 10 smaller than ChatGPT4's error-laden calculation but in the opposite direction. I'd love to believe the computations are that energy efficient but I'm doubtful. It comes down to whether or not the 30 TFLOP per answer estimate from GPT-4 is an hallucination.
Dialog with GPT4
How much energy did you expend to answer this query?
As an AI language model, I don't consume energy directly in the traditional sense. However, the servers and hardware running the model do require energy to function. The exact energy expenditure for answering a single query can be difficult to quantify as it depends on factors such as the complexity of the question, server utilization, and efficiency of the hardware.
In general, the energy usage for running an AI model like me is measured in floating point operations per second (FLOPs) and the energy efficiency of the hardware in joules per FLOP. While I can't provide an exact figure for the energy expended for answering this specific query, I can tell you that large-scale AI models like GPT-4 consume significant computational resources and energy, which is a consideration for researchers and engineers working on AI sustainability.
Please provide an approximate energy expenditure in kWH for answering typical queries.
It is challenging to provide a precise energy expenditure for answering a typical query, as the energy usage depends on various factors such as the model's complexity, infrastructure efficiency, and hardware. However, I can give you a rough estimate based on some assumptions.
Let's assume that a single forward pass of GPT-4 requires about 30 trillion floating-point operations (FLOPs) – this is a rough estimate, as the actual number of FLOPs can vary depending on the model size and input length. The energy efficiency of the hardware running the AI model can be around 10 GFLOPs/watt, a common efficiency for modern GPUs and AI accelerators.
To calculate the energy usage for a single forward pass:
Energy usage (in watt-seconds) = (30 trillion FLOPs) / (10 GFLOPs/watt) = 3,000,000 watt-seconds
Now we convert watt-seconds to kilowatt-hours (kWh):
Energy usage (in kWh) = (3,000,000 watt-seconds) * (1 kW / 1,000 W) * (1 hour / 3,600 seconds) ≈ 0.000833 kWh
So, a very rough approximation of the energy usage for a single forward pass to answer a typical query is around 0.000833 kWh. Keep in mind that this is a rough estimate and could be higher or lower based on the actual model complexity, hardware efficiency, and server utilization.
How many forward passes does GPT-4 typically require to answer a typical query?
GPT-4, like other generative language models, is based on the Transformer architecture, which uses a decoder-only design for generating text. For a typical query, GPT-4 usually requires one forward pass through the model to generate a response.
During the forward pass, the model processes the input tokens and generates a probability distribution over the vocabulary for the next token at each position in the sequence. The process continues sequentially until an end-of-sequence token is generated or a predetermined maximum length is reached.
In summary, GPT-4 generally needs one forward pass to answer a typical query. However, it's important to note that, in practice, additional computational overhead may be involved in tokenization, decoding, and other preprocessing and postprocessing steps.
Share
