“No one builds a data center to show their technical prowess, or how green they can be they build it to house IT systems reliably, and sustainably,” says Chris Brown.

That’s what you’d expect him to say: as CTO of the Uptime Institute, Brown’s job is advising the industry on how to keep data centers running. And within that role, he has some interesting points to make about the role of batteries and diesels generators.

“Batteries have been a part of data centers for decades, but they've been a short-term power source, associated with UPS systems,” he explained in a recent DCD webinar. “They have a very short ride-through time, until engine generators or some other onsite power production system can take over.”

This article appeared in our Critical Power Supplement. Read the whole thing here

Generators are here for the long haul

Power grids are normally very stable, but data centers need a long-term backup power source - usually mechanical generators - that can cover for the grid during prolonged outages, and a short-term one with two jobs: covering for brief fluctuations and, during any long-term outage, powering the data center till the generators start.

“For long-term power generation, engine generators are usually the preferred option because they pack quite a bit of power into a relatively small footprint. And as long as one can obtain additional fuel for them, they can typically provide power for that data center for a long duration,” says Brown.

Diesels may be practical, but anything which burns fossil fuel in a climate emergency is a bad thing - and there are other problems, says Sanjoy Banerjee, Professor at the City University of New York and director of the Energy Institute: “Diesel generators are hard to actually operate and locate - particularly in urban areas like New York, Singapore, or Los Angeles.

“People don't like to have diesel generators in the middle of the city. Perhaps we could replace these with batteries which have long-duration storage.”

Steve Shapiro of EYP Mission Critical Facilities isn’t so sure: “Diesel isn't the enemy. It's just how we use it, how we control it, and what we do with it.” He argues that clean diesels with emission reduction technologies can be cleaner than a fossil-based grid.

As Banerjee points out, even apparently “green” replacements such as fuel cells actually have to burn fuel: “Fuel cells are a glorified generator, unless they're using so-called green hydrogen… but a generator could also use green hydrogen or some green fuel.”

Elsewhere in the Critical Power supplement, we look deeper into the use of fuel cells, and the ways in which generators can be made greener with better fuel, for now, though Brown thinks “engine generators are going to be sticking around for a while.”

Batteries take up the slack

However that plays out, there’s surely a bigger role for batteries, with Brown predicting an increased ride-through time: “When I started in data centers, large installations had 15 minutes or up to 30 minutes of ride-through time. Over the last 10 years, I've seen people reducing this down to five minutes. I always recommend against this because once that battery starts to age a little bit, five minutes becomes three minutes - and suddenly you're you're having trouble with the battery actually doing its job.”

This is partly as a way to reduce reliance on generators, he says: “I think we're gonna see an increase in battery ride-through time to enable some data centers to be able to run a little bit longer, without having to employ an on site power production system, which typically is engine generators.”

That might leave a lot of diesel generators essentially unused: “If you look at the outage profiles, the vast majority of outages are, restricted to a couple of hours.”

Banerjee has a practical example: “A large supercomputer center that I'm dealing with has a need to go from a 10-minute to 15-minute lead-acid system to a battery system which can last two to four hours.”

It’s in a dense urban area, he says, and ”they really, absolutely don't want to use diesel generators, if they don't have to”.

Because it’s a supercomputer center, not a data center handling real-time transactions, it can actually afford rare periods of downtime. “But if it does shut down, it has to safely shut down.”

Within this trade-off, he’s taking a “defense in depth” approach: “The batteries should cover 99.9 percent of the outages.”

“The vast majority of the outages they can foresee - and the ones they've had in the last 10 years, have been significantly shorter than that. So a 1MW backup for around four hours covers their needs.”

Overcoming past failures

While the future looks good for battery storage, it’s got to overcome the memory of past failures, when batteries have not been able to cover in extreme situations. In 2011, an earthquake caused the nuclear power plant in Fukushima, Japan, to shut down. With no electricity supply, the reactors relied on diesel generators to circulate coolant - until the tsunami which followed the earthquake hit, and took the generators out of action.

“Batteries were the last line of defense at Fukushima - and after about eight hours, the batteries ran out,” says Banerjee. “And that was a big problem. I think the people in the plant actually tried to hook up their car batteries to keep it going.” Ultimately, the coolant stopped flowing and reactors melted down, creating the worst nuclear accident since Chernobyl in 1986..

Today, newer battery technologies can step up to the plate, and can cover longer timespans, with more reliable power, says Brown: “Battery technology has been advancing fairly rapidly recently. I've been in the data center industry for over 25 years, and it seems like for the first 15 or 20 years, battery technology didn't move. And now we have lithium-ion batteries, we have people looking at zinc as an option. There's all sorts of different things out there, where you can put more power in a smaller footprint than what we have seen in the past.”

And there are further benefits to new batteries, he adds: “The analysis shows that these batteries should last longer than our traditional lead-acid batteries before they have to be replaced, although we just don't have enough real installed data and a large enough installed space to really know if that's truly going to pan out.”

Banerjee thinks new batteries might handle up to 18 or 24 hour outages: “Long duration battery installations, of say 18 to 24 hours, typically cannot be done with batteries like lithium-ion, but some other technologies could do it.”

Shapiro isn’t convinced: “New battery technologies are more for UPS backup, or for direct DC power, as opposed to going through a long-term utility outage. For the next five to 10 years, we're looking at short-term battery power until some long-term system kicks in.”

What’s backing up the batteries?

Even if we have long-term battery back up, that raises a further question for Banerjee: “If the power goes out for a week, how are these batteries going to be backed up? Solar, wind, something like that? This is an open issue currently.”

Brown thinks the answer might be to use distributed processing to shift loads away, and let the data center shut down: “If I can get data centers that are far enough apart, so a single power event in one data center doesn't impact the other one, then engine generators and fuel storage starts to become a lot less important,”

In winter 2021, when extreme weather affected the grid in Texas, diesel backups were hard to maintain: “If someone had a data center just in one of the neighboring states and could have transferred the loads to that, then they might not even have had to run any sort of engine generators.”

“I think batteries are going to be a good interim step. We're going to see an increase in ride through time,” he says. “But I think the industry is also looking to become less reliant on any single data center in any single location at any single time and distribute that computing out. Instead of trying to bulletproof my data center to everything Mother Nature can throw at it. I'm going to switch over to another data center.”

Shapiro agrees: “When we look at replacing diesel generator power, you probably find the best bang for the buck in getting redundancy is on the IT side of the house - then there's less of a reliance on things like diesel generators and batteries.”

Which battery is best?

But which new battery technology is best? For Banerjee, that’s another open question: “It’s a little bit like how many angels can dance on the head of a pin.”

Lead-acid is extremely mature, he says, and actually could do longer ride-throughs: “It's the standard solution right now. It is great for 15 minutes storage. But if you go to four hours, you multiply that by a factor of 16 - and that becomes a large installation. Lead-acid has a compactness problem.”

Lead also has environmental issues: “Even though it's fully recycled, there's a certain amount of leakage in the recycle system. Another aspect we have to put into the equation, is the recycling of battery materials to have a really environmentally low impact.”

Among the replacements lithium-ion is the front runner. “Right now everybody's going towards lithium-ion,” says Shapiro. “We're doing a lot of projects right now where we're replacing lead acid batteries with lithium-ion batteries. The footprint is smaller, the way to connect it is slightly different.”

Lithium-ion has a better total cost of ownership in the seven to 15-year range, he says: “But you can spin those numbers any way you want: total cost of ownership means what it means in the eye of the beholder.”

Lithium’s safety drawbacks

In the meantime, there are still fire protection concerns with lithium-ion, Shapiro says: “You can close your eyes and say, there's no problem at all. It's got the controls and technology to shut itself off in the event of a problem.

“But we keep seeing large storage facilities that have fires and we have seen data centers that have lithium-ion batteries that have had fires. And when lithium-ion catches on fire, you really can't put it out. It's burning until it's gone.”

In Morris, Illinois in June, thousands were evacuated for four days, while firefighters struggled to contain a fire in 80 tons of second-hand EV batteries stored at Superior Battery. In August, at a 300MW Tesla battery storage facility in Australia, a 13-ton shipping container caught fire, and had to be allowed to burn itself out over four days.

Brown says: “There's all sorts of reasons why the lithium-ion battery could have a problem. Even physical damage - if you expose the lithium to oxygen it is going to erupt into a fire. For a suppression system to put it out, you're not going to dump water on it, because then that has another bad reaction.”

The Morris Illinois fire is a case in point. It burnt until firefighters finally smothered the burning batteries with 28 tons of cement.

That sort of event is causing some to pause with lithium-ion, says Brown: “People want to go to lithium-ion because it's a technology that is at least somewhat tried, true and proven. They know what they're going to get. But I'm also hearing from manufacturers of these systems, that some big clients with lithium-ion installations won't be installing any more, because insurance companies are not going to underwrite them.”

Safety codes help - but restrict usage

Safety codes are catching up, but creating a challenge of their own, says Brown: “You're able to get over it if you can put enough money into it. The insurance companies are starting to push back on it in some instances. And that's going to be a challenge.”

Those new safety codes may make lithium-ion safe, but they are changing, says Shapiro: “The codes that are coming into play on lithium-ion systems are still going to be tweaked.”

The codes have come in the form of UL 9540A testing from the safety authority, Underwriters’ Laboratories. This is designed to prevent the thermal runaway fire propagation which is the bane of lithium-ion.

Shapiro says: “The adoption of new codes is restricting how you use lithium-ion, unless the lithium-ion cabinets are UL listed and applied in certain ways.” Ultimately safety rules will require a system to disconnect every single battery in the string to minimize the impact of a potential fire, he says: “but I don't think that technology is there yet.“

This could mean that data centers in cities cannot use lithium-ion batteries for backup, warns Banerjee: “The fire codes for lithium-ion are becoming more and more stringent. In New York City, FDNY [the Fire Department] requires that you go through 9540A UL certification to prevent a thermal runaway. Currently, I would say it's virtually impossible to site a lithium-ion system within a building in New York City. And I don't see that situation changing in the near future. I've been trying forever.”

Sites that really want lithium-ion batteries search for loopholes, swapping batteries to get past the Fire Department rules, or placing them in containers on the street or somewhere the rules don’t apply. As Shapiro points out, that doesn’t take away the fire risk: “When that container catches on fire, there's no fire protection inside the box. And there's no way for the fire department to do anything about it, it has to burn itself out.”

Artificial pricing

Even in places that allow lithium-ion systems, restrictions could obstruct the development of newer data center architectures, says Brown: “We’ve had a big push in the data center industry to try to distribute our UPS systems and distribute battery systems [sometimes into the racks] instead of having the large monolithic UPS systems in a separate room.”

Unfortunately, the safety concerns over lithium-ion are pushing data centers back to those monolithic battery rooms, because of the risk of a fire inside the IT space.

There’s another long-term concern with lithium-ion: its current attractiveness may be based on an artificially low price, says Banerjee: “Lithium costs are distorted. Currently, there is a vast overcapacity for lithium-ion phosphate in China, leading to artificially low prices for lithium-ion phosphate batteries in particular. This move towards lithium may be engendered to some extent by the very large Chinese capacity, which was developed under a subsidy.”

For all these reasons, Brown thinks data centers will eventually move to other technologies: “Lithium-ion is a step in the right direction. But I don't think it's something that we're going to be reliant on for decades to come.”

Zinc - an aqueous option

Beyond lithium-ion, there are a few options, says Shapiro: “There’s a nickel-zinc system, but it hasn't been fully indoctrinated into the market yet. There's also sodium systems, but sodium isn't quite ready for Megawatt UPS capacities in the markets that I'm seeing.”

Zinc has big potential, says Banerjee: “It’s an aqueous solution, and you can't take an aqueous zinc-based solution to thermal runaway. So the whole 9540A issue more or less vanishes.”

And Banerjee has practical experience that shows it can be made to work acceptably: “I personally have installed in City University, a 200kW hours system, based on zinc and nickel. The FDNY permitted that. It was fully permitted.”

It was an experimental system, however, and has now been dismantled after five years operation, he says “But it certainly can be done. We know that, just like lead-acid, we can permit zinc-based systems, because UL people can't take them to thermal runaway.

Alongside this, “zinc-based systems can be pretty much as energy-dense as lithium and, because it doesn't need a lot of cooling systems, it also could be comparable in cost.”

Sodium batteries share a lot of the benefits of zinc - they are aqueous, less risky and potentially have a high power density. Sodium is also a cheap and plentiful element. But like zinc batteries, they are at an early stage still.

The backup discussion ends in a conundrum. We need batteries more than ever, to displace fossil fuel based solutions and provide reliability, but our main current technology is bulky, the lead contender to replace it is risky, and the alternatives beyond that for an eventual replacement are still under development.