In debates around using LLMs and AI models, there’s a certain kind of argument that comes up again and again: people will cite the use of natural resources by the data center servers that are crunching all of these demands.
Now, there are two different ways to come at this type of evaluation: you can try to assess it on a micro level, or a macro level. You can ask: what is the water and energy use of a single query, or you can ask: what is the total appetite of the systems running the models themselves?
Every Time You Tap
Just a cursory look at the first method shows you how subjective and vague it is.
You get anything from over 500ml per GPT-query, all of the way down to .3ml (or 1-1666.67th of the first number?), a figure attributed to Sam Altman, depending on things like whether you account for upstream cooling, not to mention what the task is, and where the demand is coming from. There are so many factors, in fact, that it really becomes, to some extent, an exercise in futility, according to quite a few experts who suggest we focus on making things more efficient.
The Big Picture
If you look at total use, you may see a more defined picture, although to be fair, it’s a complex landscape.
I was listening to a talk by Yankai Jiang, sustainable data center design researcher, at Planet Action this year (an event that I help to run,) where he estimated total data center consumption at around 4.5 terawatts, or as he put it, “7 New York Cities” of power.
“Nowadays, we are surrounded by data centers,” Jiang said. “Every email we sent, every photo upload, every binge watching night. Somewhere a data center stays up late for you.”
He also personified these systems quite a bit, which I want to include here for the reader’s edification, a kind of thought experiment for our times.
“If data centers were human,” he began, “they would be workaholics. Always awake, always online, and always overheating. They never take vacations. They never sleep. And the more we depend on them, the more exhausted they become. So think about it. They eat electricity. They drink water. And when they are stressed, they sweat a lot. But the only difference is when humans burn out, we go to see therapy. But when data centers burn out, we just retire the old ones and build new ones.”
It’s evocative, especially in an age where we’re suddenly trying to figure out just how “human” the systems are. Not the data centers, but the non-deterministic models they are supporting.
Anyway…
Jiang came up with some good ways to promote sustainability for data center designs, making the argument that common-sense changes will matter.
Put Them in Good Places
Contrasting data center plans in Arizona versus Finland, Jiang showed how environmental factors apply.
“Dave, who lives in Arizona, is hot, dry and (has) water stress every day,” he said. “He consumes huge amounts of electricity and water to stay cool. On the other hand, Dave, who lives in Finland, he takes advantage of the cold climate and nearby seawater to cool his servers naturally, without using a single drop of fresh water. And when renewable energy peaks, he starts to run his jobs. That’s what good design looks like.” (Here’s more on data centers around the world.)
Carbon scheduling, he noted, is only part of the equation, but it’s a good start. Another thing he recommends is to combine hardware according to need.
“You do not always need a Ferrari,” he said, in an automotive analogy. “Sometimes a reliable Toyota will do just fine.”
A Job We Do Together
Broadening the aperture, Jiang suggested that this goal of making things more efficient and sustainable is a community effort.
“Building sustainable data centers is not just a job for the people who run and manage data centers,” he noted. “Sustainable computing still has a long road ahead. We need more people to join this effort, because the earth is not just where our data centers live. It’s also our home, and it’s our hope.”
That’s a little bit about how the water and energy use shakes out. People keep trying to game this out more concretely, but the numbers are slippery.
“Aggregate disclosures are more reliable than per-query math, because companies report them directly,” writes Jordan Hale at Tech Journal. “The numbers are large and rising.”
If we can make those numbers do what we want them to, through the power of design, we’ll come out ahead. Stay tuned.
