Rachel Moorehead is used to outages. For the past five years, she’s had to keep a pair of ancient data centers online for the University of Alabama at Birmingham, a task that has proved exhausting.
“Power outages, generator failures, chilled water disruptions and cooling emergencies, happen about every four months,” she said at DCD>San Francisco.
“And that is enough to make anybody have grey hair, and I’m not 40 yet, and I don’t want grey hair, so we need to find a way to fix that.”
Now, the university plans to launch a new supercomputer in a brand new building with all-new equipment. Finally.
This feature first appeared in the 2020 Power supplement. Read it for free today.
“It’s not quite done yet - pretty much all of the exterior is done, and so we’re all working on the inside now,” Moorehead said. “We’re setting up the data center now, the racks are here, we’re getting in the containment systems. It’s all coming together and we’re super excited.”
Rather than building a standard facility, the UAB’s Innovation Center looked to live up to its name, and try something a little different. But it also had to occupy a small footprint, be affordable, meet the university’s sustainability goals, and last up to 50 years.
To help pull this off, the university turned to design consultants RSP Architects. “Before we actually put any paper to pencil, or designed on CAD, we really had to understand what the owners’ requirements were to determine what the product will look like,” RSP’s Rajan Battish explained.
“The fundamental requirements were basically a Tier II data center, because this is a university data center. We’re not looking to build a financial bank Tier IV type data center solution,” he said.
“In addition to that, we needed to make sure that the operations and maintenance of the IT equipment and the infrastructure that supports it provided a 2N solution because that’s where most of the human interaction occurs. And typically, most of the failures occur downstream of that critical system.”
The site was fortunate to have power coming in underground, with three 15kV circuits connecting to a series of network transformers in a vault.
“And we have a network transformer configuration feeding the data center at an N+1 configuration,” Battish said. “If you can get a solid stable utility source, it basically helps increase your resiliency to the facility, and also the reliability.”
The university could have fed that power into a traditional data center set up with UPS systems and generators, but were already burned by the experience on their existing facilities. “We have the traditional model of UPS and generator, and we knew that didn’t always work for us,” Moorehead said. “There were just too many times we had the same component fail on both legs of our power that we wanted to make a wise choice.”
The team wanted to diversify but “we didn’t want to go too much off the reservation,” she said. The result was to use a traditional generator-UPS model for one leg of the facility’s power, and on the second leg, we wanted something with fewer moving parts” that was less likely to fail.
“And so then stage-right, enter the BESS system,” Moorehead said. What she calls the ‘best energy storage system’ is a battery energy storage system featuring Tesla Megapacks converted to work as backup power in case of critical power failure.
“The Megapack is an all in one system,” Tesla’s senior project development manager Jake Millan said.
“It’s completely pre-assembled and pre-tested in a single enclosure from our Gigafactory in Nevada - This includes battery modules, bi-directional inverters, thermal management systems, an AC main breaker and controls. There’s no further assembly required on site. All you need to do is connect the Megapack’s AC output to your site wiring.”
But to actually get the battery system fit for a data center required a little bit more work, Battish said. The three 15kV power circuits came via a “network transmitter sitting outside to a switchgear that was designed to accommodate all three circuits at 40V and the power flow actually comes through a static switch that is a fast disconnect that we needed to provide to adapt the BESS UPS system for commercial data center application,” Battish said.
“In addition to that, we ended up putting in galvanic isolation between the utility source and the BESS system by putting a Delta Y transformer.”
Power comes in through A and B feeds, with BESS serving as the power conditioner and standby power for A, and the traditional UPS serving as power conditioner for B, while a generator is used as B’s standby power.
“You’ve got a battery on one side, and you’ve got a traditional generator-UPS system on the other side, “ Battish said. “It provides that extra level of resiliency to maintain uptime, not just electrically, but mechanically as well.”
The work required to make this possible was considerable, Battish said. “Anybody can do this, but there’s a lot of thought process that needs to happen for this to work.”
While much of the technology in BESS and standard UPS systems is quite similar, “the controls and the software can be quite different,” he said.
“That’s the key: how do we get those two different technologies to work properly in the output so that there’s no outage to the critical load. That was something we had to flush through during the design,” he added.
“Also the sequence of operations. When we have all these cross ties between A and B sources of future loads, we have to be very mindful of how do we develop that sequence of operation, the failure modes associated with the sequence of operations and, obviously, the cost.”
Now, the team is planning to add a third leg of power, with another BESS system, perhaps using Tesla batteries, but potentially someone else’s.
“We definitely do not want to be sole-sourcing to Tesla solutions, per se,” Battish admitted. “We want to look at different manufacturers, and see if the market will garner that desire and the interest to create that technology.”
The idea of big battery storage solutions is not new, he noted, pointing to the hundreds of megawatts being installed around the world. “But it’s a new application, and it challenges the status quo.”
He continued: “It’s like, why not? Why can’t we do this?”