“Generative AI is the defining technology of our time,” declared never-knowingly understated Nvidia CEO Jensen Huang at his company’s GTC developer conference back in March. “Blackwell GPUs are the engine to power this new industrial revolution.”

Huang was speaking at the launch of Blackwell, the latest GPU architecture designed by Nvidia to train and run artificial intelligence systems. As is customary, it offers hefty performance and efficiency bumps when compared to its predecessor, the Hopper series.

But packing more and more transistors (the B100 GPU features 208 billion, compared to 80 billion on the previous generation H100) onto a single chip comes at a cost. Nvidia says the first devices in Blackwell series require 700W-1,200W of power each, up to 40 percent more than the H100, and features direct liquid-cooling capability for the first time.

For data center operators, growing demand for high-density servers featuring AI-focused processors from Nvidia and other vendors presents a challenge: how to keep cabinets full of GPUs cool as their power requirements increase and they generate more and more heat.

For many companies, this means redesigning their systems to cater for AI-ready racks, utilizing emerging liquid cooling techniques, which are proving more effective than traditional air cooling. Whether generative AI truly turns out to be the defining technology of the current era, or a triumph of hype over substance, remains to be seen, but it is certainly redefining how data centers approach the cooling conundrum.

Hot chips

While much talk in the industry is of racks up to 100kW in density, the reality in most data centers is somewhat lighter. JLL’s 2024 Data Center Global Outlook report, released in January, shows the average density across all data center racks to be 12.8kW, rising to 36.1kW when just taking into account hyperscale facilities, where most AI workloads run.

However, the report expects this to rise to an average of 17.2kW across all data centers by 2027, hitting 48.7kW for hyperscale facilities.

With technology evolving at a rapid rate, data center operators are having to grapple with the challenge of being prepared for a high-density future, while also meeting the day-to-day needs of their customers. “We have to be all things to all people,” says Tom Kingham, senior director for design at CyrusOne.

“We’ve seen the recent announcements from Nvidia around the Blackwell chipsets, which are liquid-cooled and have incredible densities. So we know that’s coming, and the challenge for us is that the data centers we’re designing now will not go into operation for another three years.

“We need to have the flexibility to know we can deal with a client requirement now and be able to deal with it in three years’ time when the thing goes live. And because we want a 10-15 year lease term, we would like it to at least be somewhat relevant by the end of that end of that term as well.”

CyrusOne, which focuses on providing data center services to the hyperscale market and is backed by KKR and Global Infrastructure Partners, has come up with a solution it calls Intelliscale, a modular approach to building data centers. The company says it will be able to accommodate racks up to a hefty 300kW in density.

To cope with the cooling demands of such racks, the system can be kitted out with a mixture of liquid cooling technologies, encompassing direct-to-chip liquid cooling (DLC), immersion cooling, and rear door heat exchange, as well as traditional air cooling. This is a “natural progression” of what the company was doing already, Kingham says. “We've been using modular electrical plant rooms, packaged chillers, and cooling products that form 1.5MW blocks,” he explains. “We put as many of those blocks together as we need to get to the capacity of the data center.

“All that we've done with Intelliscale is add in some additional components that provide us the flexibility for the data center to be liquid- or air-cooled, or to use things like immersion cooling.”

Kingham says CyrusOne is currently hedging its bets when it comes to which types of cooling technology will be favored by customers and equipment vendors. “At the moment, we have to offer everything,” he says. “But it feels like DLC is the preferred method at this point. With that, comes an element of a hybrid set-up where the chip will be cooled by liquid but other components, like the power supply unit, still need some air.”

A liquid future

Other vendors are also turning to liquid cooling in a bid to cope with demand for high density. Equinix announced in December it was making liquid-cooled solutions available in more than 100 of its data centers around the world, without specifying the kind of densities it was able to handle.

Stack Infrastructure launched a high-density offering in January, saying it could support up to 30kW per rack with traditional air cooling, up to 50kW per rack with rear door heat exchangers, and up to 100kW per rack with DLC. The company said it intends to support 300kW or higher per rack with immersion cooling in future.

Aligned is promoting a similar product, known as DeltaFlow, which it says can support high-density compute requirements and supercomputers, and can cool densities up to 300kW per rack.

Digital Realty’s high-density solution hit the market last year, supporting up to 70kW. In May, the company revealed it was more than doubling this to 150kW, and adding DLC into the mix. This is now available at 170 of the company’s data centers, representing just under half of its portfolio.

“We already have really strong demand for AI and HPC workloads across a variety of business segments,” says Scott Mills, SVP for engineering and customer solutions at Digital Realty. “That has ranged from sectors like minerals and mining, to pharma and financial services. At cabinet level, we’re seeing densities going up from 30 cabinets to 70, and within those 70 cabinets you might have 500kW of you might have 10MW.”

Mills says that for Digital’s platform to be able to cope with all these different densities effectively, it’s important to be able to “plug in” new cooling systems like DLC. He says the company intends to continue expanding its offering: “We’re going to keep doing this, and the plan is to make the platform broader as new technologies become available,” he says.

Up on the roof

Making new cooling technologies available is one thing, but fitting them into existing data centers is a different challenge entirely.

CyrusOne’s Kingham says the company is currently grappling with the challenge of retrofitting its Intelliscale units into a data center it designed only a year ago. “We're already seeing customer demand to convert that into a liquid-cooled facility,” he says. “Principally, it's not very difficult, because we were already running a closed loop water system into the building, but the densities are so much higher that our challenge is getting enough heat rejection equipment into the building.”

Heat rejection systems are typically placed on the roof data center, which is all well and good unless that roof space is shrinking. “Our initial hypothesis was that we could make the data centers smaller because the racks are getting denser, therefore we could make the rooms smaller,” Kingham says.

“But that means we don't have enough roof space to put all the heat rejection plant. And we don't necessarily have enough space in the building to coordinate all the power distribution for those racks, because now the power distribution is much larger. So we're actually finding that the rooms aren't necessarily getting any smaller but the focus is now on trying to efficiently get heat out of the building.”

Elsewhere, Kingham says making liquid cooling cost-effective (“there’s so much more equipment in this sort of design”) is also a challenge for his team, while new hazards are also emerging. “We're moving the cooling closer to where the heat is, so there's less of a buffer and, if something were to go wrong, it's going to go much more wrong now than it ever has before,” Kingham adds. “And on the other hand, if the water doesn't get to the chip, it's going to cook quickly.”

From an operational standpoint, the presence of liquid cooling is changing the way data centers operate, Digital Realty’s Mills says. “There are new service level agreements and new things we have to monitor and manage,” he says. “Something like leak detection has to become core to the offering whereas before you didn't really talk about it before because it was heresy to even think about introducing liquid into the data center environment.

“Our global operations team is developing new methods and procedures around monitoring, both for our teams and also so that we can supply alerts directly for our customers. We’ll get better at doing that, and also the industry as a whole will improve as these things become standardized.”

The liquid cooling journey

Mills says many of Digital Realty’s customers are also on a journey of understanding about how liquid cooling is relevant to their data center workloads. “We have a group of customers who have experience in doing this, who will come to us and know what they want and how they want it,” he explains. “Then it’s just up to us to work with them to make that happen.

“There’s a second group who says ‘we are learning and we want your expertise to help us,’ and that’s where we have to provide more support through our global programs.”

It’s a similar story at CyrusOne. “We’re all learning together,” Kingham says. “It’s easy for us, as the operator, to get frustrated because we have to set things in stone now that will go live in 2027. But when you look at how much change has happened in the last year, even with the products announced by Nvidia alone, what is the picture going to look like in three years? It could be could be totally different.

“I think the real challenge is going to come in the next tech refresh cycle, in the early 2030s. What we’re designing for now may be completely obsolete by then.”

Regardless of what the future holds, Kingham believes the advent of AI has accelerated adoption of liquid cooling, and that the change is a permanent one.

“We'll see how the market trends over time, but, from my point of view, all the discussions now are around liquid cooling,” he says. “There's no sense that this is just a buzzword anymore, and it feels like pretty much everything we do from now on is going to be down this path.

He adds: “I wonder how long it will take us before we just pivot to accepting that, right from the very start, all our projects are going to be liquid-cooled data centers.”

The feature first appeared in the latest DCD Cooling supplement. Register here to read free of charge.