Will there be some promising techniques that can make AI greener and affordable in the future?

Question

The recent advances in machine learning were mostly achieved by the hardware, and the hardware is said to continue driving the development of AI, but I was still shocked by this thread which reads that the projected future cost for the largest model would be 1B dollars in 2025. And I learned that universities are suffering from an academic AI brain drain partly due to the scarce hardware resources.

Some people proposed the so-called Green AI that encourages sustainable AI development but provides few constructive methods to prevent the trend.

I wonder if the redder and redder AI would be in fact truly inevitable. It seems to me that all companies should build an expensive compute infrastructure to be competitive, but I think the investment would be very risky since most companies cannot get a higher return.

But on the other hand, we human beings have evolved tens of millions of years or billions of years(life) on earth with hundreds of billions of brains that have ever lived on earth as a whole "human brain". The biological wetware seems much much redder than the nowadays hardware and has consumed much much more energy than all the supercomputers. To make machines as intelligent as we humans shouldn't we pay as high a price? It reminds me of the NFL theorem but it should be imprecise in this scenario.

So, will there be some promising techniques on the algorithm side that can make AI greener, affordable and sustainable in the future? If not, could anyone please explain why AI should be unavoidably red and inevitably redder?

Answer 1

As far as I know, green AI and red AI are very recent terms and/or research areas, but their importance will or should definitely be more highlighted in the coming years (for obvious reasons).

I don't know how many people are actively and directly researching this topic, but, in the past, I've already read a research paper on the topic, so a few people are already trying to at least raise awareness about these issues. Moreover, they also suggest that researchers report the number of floating-point operations (FLOPs) performed during the experiments, in addition to the usual performance metrics (such as the accuracy). I don't remember all the details of this paper, but I really recommend that you read it. This another possibly useful paper that people interested in this topic might want to read.

Although I'm currently not doing research on this topic, from an algorithmic perspective, improvements in the following areas will probably contribute to greener AI

• zero-, one- or few-shot learning
• transfer learning
• sample/data efficiency
• computational complexity

If you can learn from fewer samples and with less computation, you will probably produce less emissions.

So, I think that making AI greener is definitely possible, as it's possible to use an electric car rather than a petrol/diesel car, although more research on the topics mentioned above (and other topics) needs to be done to make it practically useful.