In May 2024, DCD hosted a Management & Operations Innovation Day focusing on advancing data center visibility and operational efficiency. Industry experts discussed challenges and opportunities related to power distribution, cooling systems, and sustainability, particularly in the context of AI workloads.

Cutting-edge data center design

During the Innovation Day, Joe Reele from Schneider Electric, Joe Hubbard from AVAIO Digital Partners, and Peter Hopper from Digital Bridge discussed the complexities of designing cutting-edge data centers.

They emphasized the need for a collaborative, multidisciplinary approach involving finance, IT, operations, and strategic planning to address project specifics effectively.

Reele highlighted this collaborative approach, stating:

“It takes a village to raise a child, and the solution involves a multidisciplinary approach. ‘The village’ includes finance, IT, operations, and strategic planners. Each group brings essential insights to the table, helping to make intelligent decisions about the what, why, when, and where of the data center project. There is no one-size-fits-all solution.”

Key challenges of cutting-edge data center design

The discussion also focused on key challenges such as sustainability, adaptability, and collaboration in data center design. Hopper advocated for integrating water delivery directly to racks for efficient heat management, stating:

“We believe that data center designs should include water delivery to the rack. This could mean taking water directly to the chips to evacuate heat, which is becoming more common for racks exceeding 100 kilowatts each. Alternatively, bringing water to the top of the rack can generate cold air at the source. This flexibility allows for a mix of low-density and high-density racks, such as networking gear next to high-performance servers.”

We look for facilities that have these features already integrated, as retrofitting is extremely challenging. It’s like changing the engines on an airplane while flying – possible, but not advisable.”

Hubbard explored evolving energy landscapes, noting:

“When aiming to avoid obsolescence, it's crucial to find a campus with the land to bridge the gap and go online quickly to support customers. Things that used to be taboo in the data center world from a risk management standpoint are now becoming desirable. For example, being close to a gas line is now advantageous.”

The growing importance of renewable energy sources

Speakers highlighted the growing significance of renewable energy sources and advancements in energy storage technologies. Reele explained:

“Today, data centers typically rely on a simple energy input model – either utility or generator-based – converting AC to DC and back. But the world is moving towards a “prosumer” model, where data centers can produce their own power. Now I can match the quality, quantity, and type of electrical utility to the criticality and quality of the load.”

He emphasized the challenges posed by variable energy sources like solar and wind, stating:

“For instance, solar and wind energy are variable – solar doesn't shine consistently, and wind doesn't always blow predictably. Advanced batteries help minimize power loss during these fluctuations. This variability in utility presents challenges; for instance, during rainy days, solar output may drop, potentially causing a 200-kilowatt loss. But with intelligent IT systems, such as those capable of redistributing energy to other areas of need, we can effectively manage these fluctuations.”

Hopper added:

“It’s no longer sufficient to rely solely on offsetting dirty electrons consumed on campus with clean ones injected elsewhere. The demand now is for clean electrons directly on campus.”

“It’s crucial to remember that data centers, like all consumers, draw power from the grid. As data centers grow, they may face increasing scrutiny and public pushback during peak demand periods, highlighting the need for careful energy management and sustainable solutions.”

Educating the public about the benefits of data centers

Hubbard emphasized the importance of public education about data centers, stating:

“Trying to get the education out there about how long it has been around, and how long it's been used in different applications that people just don't realize. But also as data center providers and developers, we need to come up with ways to put back into the community.”

Developing AI infrastructure: Design considerations for critical workloads

In a subsequent session during the Innovation Day, DCD’s Stephen Worn sat down for a 1:1 with Robert Bunger of Schneider Electric, to discuss the growing demand for power-hungry processors in AI and the increasing importance of liquid cooling systems. Bunger started off by acknowledging the ongoing trend of increasing processor power consumption and the critical role of liquid cooling systems in managing heat:

“This trend is not new but is still noteworthy. Processor power levels and heat flux are continuing to increase, which intensifies the need for effective cooling solutions like liquid cooling or direct-to-chip cooling.”

Bunger emphasized the role of cooling distribution units (CDU) for high-density AI servers, whereby efficient cooling solutions like liquid cooling are essential to maintain optimal performance and prevent overheating:

“CDUs provide the right temperature for IT equipment, correct flow rates, and meet pressure requirements, making sure that fluid treatment, water chemistry, and filtration integrity are maintained. Last but not least, CDUs facilitate efficient heat exchange, crucial for sustaining server performance.”

“As you surpass 50 kW per rack, the primary concern shifts to the heat flux at the chip level rather than simply the heat per rack. This transition underscores the discussion we’ve been having about liquid cooling, which has been ongoing for decades, particularly in HPC environments like national labs. Now, driven by AI applications, this technology is making its way into mainstream data centers.”

The benefits of optimizing data center design and operations

Transitioning to liquid-cooled data centers, Bunger discusses the initial costs and long-term benefits:

“Moving towards liquid cooling entails upfront costs, particularly with more complex rack setups involving cold plates, dripless connectors, and fluid manifolds. But when we break it down component by component, the efficiency gains become clear.“

“For instance, liquid-cooled racks typically use less energy due to reduced IT equipment fan usage, translating to potential savings of up to 15% at the compute level alone.”

“As the industry standardizes liquid cooling practices and technologies, I foresee a future where liquid-cooled data centers not only match but potentially outperform air-cooled counterparts in terms of TCO and operational efficiency.”

The discussion underscores challenges and solutions in liquid cooling adoption, highlighting the need for meticulous design and infrastructure planning to optimize data center performance.

Evolving powertrain design to better support AI workloads

During the third session of the Innovation Day, DCD’s Kat Sullivan discusses the need to evolve power design to support AI workloads with Schneider Electric’s Stuart Sheehan. Sheehan began by emphasizing the rapid growth in AI demand and the need for agile responses:

“With AI, there's a degree of whiplash in meeting the rising demand in forecasts, requiring blocking and tackling just to keep up with the demand. But an important aspect is considering how AI workloads can reshape data center design. It’s necessary to keep up with demand, but there’s also an opportunity to do things a little bit more efficiently. From my perspective in the technology office, it's about balancing present-day requirements with a forward-looking approach.”

Impact of AI workloads on data center design

AI workloads offer a scalable and uniform build-out for data center design, allowing for more optimized interfaces and innovation. Sheehan highlighted the transformative impact on data center design:

“In terms of data centers designed to host AI workloads and applications, changes at the rack level serve as a catalyst for making generational changes in designing data centers.“

“This dynamic encourages innovation by pushing the boundaries of what is traditionally done in data center architecture.”

“Fast forward to today, advancements in rack designs and the demand for high-performance chips themselves, have accelerated the adoption of liquid cooling, fueled by the need to optimize performance.”

Sheehan underscored AI workloads’ predictable scalability, contrasting with legacy data centers, which must adapt to diverse demands. This predictability offers opportunities for specialized designs tailored to AI applications.

The impact of technological advancements on facility design and operations

As the conversation progressed, Sheehan recalled Schneider's Technology Office’s use of three horizons to categorize technological developments by time, scope, and complexity of changes needed:

“Horizon One represents incremental evolutions of existing technologies, so adapting to some small changes, but using the portfolio of technologies and products that already exist. That's kind of now through the next three years-ish.”

“Horizon Two involves more substantial changes across different product lines or systems. This spans beyond the next three years up to around 2030.”

“Beyond Horizon Two lies longer-term disruptions requiring substantial technological maturation. It’s not just the coordination between two different products, you need full R&D efforts as they involve significant de-risking and development to achieve technological readiness.”

Holistic design and better standardization in deploying AI applications

Finally, Sheehan outlined how to standardize and improve energy solutions for data centers:

“We see a natural progression towards more powerful racks and larger clusters, which inevitably strain existing facilities, sometimes reaching hundreds of megawatts.“

“Today, deploying large-scale infrastructure faces regulatory and construction hurdles, hindering rapid expansion. Consequently, there's a growing need to adapt to dynamic power solutions and explore 'bring your own power' options where viable.”

“Looking further ahead, as AI applications diversify, standardized and scalable energy solutions will be critical. These solutions must adapt to grid constraints and intermittent energy sources while fostering collaboration across technology providers to ensure robust and sustainable data center deployments.”

Maximize efficiency and reliability with Schneider Electric’s AI solutions.