Large language models (LLMs) eat electricity at an alarming rate. As these popular AI models continue to grow in size and popularity, their energy consumption threatens to outpace our ability to power them sustainably. This presents a looming crisis at the intersection of technological progress and environmental responsibility.
Can we do anything to stop it?
The scale of the problem
LLMs require massive computing power, both for training and inference. A single training run for a model like GPT-3 consumed an estimated 1,287 MWh of electricity - equivalent to the carbon dioxide emissions from 550 round-trip flights between New York and San Francisco. Inference, the process of actually using these models to generate responses, compounds the issue. ChatGPT's monthly energy usage for inference alone may range from 1 to 23 million kWh.
Proponents of LLMs often tout improvements in energy efficiency as a solution. They point to techniques like quantization and knowledge distillation that can reduce model size and power consumption. However, these incremental gains pale in comparison to the exponential growth in model size and usage.
This thinking is a fallacy. While smaller models like Guanaco can achieve 97% of ChatGPT's performance with less computational resources, it fails to address the fundamental issue: the relentless push for ever-larger models and wider deployment.
The demand for LLMs has sparked a GPU arms race among tech giants. Companies scramble to secure vast quantities of high-performance GPUs, driving up costs and energy consumption. This frenzy exacerbates the environmental impact of AI development. The Wall Street Journal notes that the "AI boom is fueling an insatiable appetite for electricity." While the article overestimates the immediate impact of AI on energy demand, it correctly identifies the shift towards fossil fuels to meet this growing need.
Consume, consume, consume!
As demand for AI computation grows, tech companies increasingly turn to fossil fuels, particularly natural gas, to power their data centers. This reverses years of progress in transitioning to renewable energy sources. The rush to build and expand data centers outpaces the development of clean energy infrastructure. The timeline for building a data center spans just a few years, while adding renewable energy to the grid faces longer delays due to permitting issues. This mismatch incentivizes the use of readily available fossil fuel power sources.
Beyond electricity consumption, LLMs place a significant burden on water resources. Data centers require enormous amounts of water for cooling. A recent study found that ChatGPT uses approximately 500 milliliters of water for every 50-100 queries - roughly equivalent to a standard water bottle. The study cites figures released by Google and Microsoft showing spikes in their water usage attributed to AI work — as LLMs proliferate, their water consumption could strain local resources, particularly in drought-prone areas.