Last year, Data Center Alley nearly ran out of power. This area of Northern Virginia is home to more data centers than anywhere else in the US and the local utility Dominion Energy warned it might not be able to meet surging demand.
This happened before a ChatGPT-inspired AI boom that is set to double or triple the amount of energy consumed by new types of data centers. What's coming will probably make the Dominion Energy debacle look like a walk in the park, according to industry executives, researchers, and analysts.
Major cloud providers including Amazon, Microsoft, and Google are racing to meet demand for generative AI, and access to electricity could become even more important than clever new model designs or computing heft. What's clear is that today's data centers are not built to support this new era of technology.
"It's staggering," said Marc Ganzi, CEO of DigitalBridge, an investment firm that owns and operates data centers, fiber networks, and other tech infrastructure. "Not sure how we do it."
By 2030, data centers are expected to reach 35 gigawatts of power consumption annually, up from 17 gigawatts last year, according to McKinsey. A recent Cowen research report estimated that AI data centers could require more than five times the power of traditional facilities.
The coming competition for energy resources
The tech industry, and society in general, is not ready for this new energy-intensive wave. But there are early signs of a troubling future where Big Tech builds AI models that disrupt companies and put people out of work -- all while competing against them for resources and potentially raising their energy bills.
Earlier this year, weapons manufacturer Nammo said it couldn't expand a factory because a data center used by TikTok was slurping up all the spare power in the area, according to The Financial Times.
This is "a taste of things to come, as data centers supporting great money-printing machine minds compete with other big industries for electricity," Jack Clark, a co-founder of Google-backed AI startup Anthropic, wrote in a recent industry newsletter.
AI model training is especially energy intensive. It requires the use of graphic processing units. These GPUs are specialized chips that multitask better and work faster than central processing units, or CPUs, which run most traditional cloud services.
AI computer servers being installed in data centers are often equipped with multiple GPUs, usually supplied by Nvidia. Each GPU consumes up to about 400 watts of power, so one AI server can consume 2 kilowatts. A regular cloud server uses 300 to 500 watts, according to Shaolei Ren, a researcher at UC Riverside who has studied how modern AI models use resources.
For instance, the power consumption of Nvidia's new GH200 server cluster is about two to four times more than a regular cluster of the same physical size, he estimated.
A warning from a Microsoft data center veteran
Tom Keane, who oversaw Microsoft's cloud data centers for about two decades, recently warned about this. Microsoft is the main backer of OpenAI, creator of ChatGPT and builder of the most powerful AI models around.
"In the case of a training data center, you make that as big as possible, you put as many computers in there as possible and you're running that data center at full utilization all of the time," Keane told Bernstein analyst Mark Moerdler in a recent interview. "That physical data center starts to become more resource intensive. You start to design that AI data center very differently."
An AI data center will need up to three times more power than a traditional cloud facility, he estimated. A regular setup might have 30 kilowatts powering a rack of cloud computer servers, while a new AI version of this will need 100 kilowatts, he estimated.
"Everything begins to change when you think about what AI represents," he added. (Keane left Microsoft last year, about a month after an Insider report revealed he was verbally abusive to some employees. At the time, Keane didn't respond to requests for comment.)
A 'massive' opportunity
The surge in AI demand, and the changes coming, present an opportunity for investors like Ganzi. "Massive would be the understatement," he said.
This year, DigitalBridge is focused on building and upgrading data centers to handle generative AI workloads, deploying nearly $8 billion to meet this new demand, Ganzi told Insider.
One key difference between traditional data centers and new AI facilities is how power is consumed. Regular cloud data centers are designed to handle variable demand. When everyone in the US is working and streaming videos, these facilities are busy. When night falls and most people sleep, usage falls. In contrast, AI data centers run constantly, especially when training big models. That requires increased access to power and the ability to sustain it consistently.
'The data center of the future is not in Virginia'
For companies like DigitalBridge, that means going outside existing major data center markets to build a new generation of facilities that will power the AI boom.
At DataBank, a data center provider backed by DigitalBridge, Ganzi is putting smaller data centers in the suburbs of "tier two and tier three cities," where the electricity grid is established but not stressed by existing energy-sucking cloud workloads.
Proximity to smaller urban areas means better power density, which means a faster connection. Smaller cities also mean lower power costs for the data center operator. Ideal locations for future data centers will be in low-cost areas, according Cowen.
"The data center of the future is not in Virginia, it's not in Santa Clara, it's not in Dallas, Texas," Ganzi said. "Those are the major markets right now, and there's a limitation of the amount of power you can get in those markets."
Do you work at a major cloud provider? Do you have insight or information to share? Contact Ellen Thomas on Signal at 646-847-9416 or ethomas@insider.com using a nonwork device.
