Data centers are supposed to be available all the time, but the power grid sometimes fails. Facilities traditionally have an uninterruptible power system (UPS), which ensures that they keep going when the grid does not.

Typically, the UPS consists of diesel generators that fire up when needed, and deliver enough power to keep the data center running. The UPS also includes switches, and rooms full of batteries that store enough energy to keep the facility up till the diesel is running, and can support it.

This feature appeared in the October Critical Power Supplement. Read it for free today.

Data center operators are all committing to become carbon neutral. They use renewable energy for their regular power needs, and are now turning their attention to replacing the fossil power in their backup systems.

Why do it?

It’s worth putting this in context. In most developed countries, power outages are rare and shortlived. So diesel generators are only fired up rarely. If there are no outages, they will be used for a total of a few hours per year, to test them.

If the operator does a full load test for half an hour, every couple of months, that amounts to a solid amount of energy used, with associated emissions. But it’s a tiny fraction (less than a thousandth) of the energy used by the data center as a whole over that period.

However, if the operator switches its regular power to a renewable source, then the emissions from backup tests will remain stubbornly static, and become a (slightly) larger proportion of the overall emissions caused by that site.

Another factor is that diesels are unpopular and highly visible. Unlike the grid energy used to power the facility, the generator can be seen at the site, along with its tank of fuel, and any noise and emissions will be noticed by the public.

Ten years ago in 2012, Microsoft suffered a humiliating campaign against its Quincy data center, led by former Mayor, environmental campaigner Patty Martin. The campaign was taken up by the New York Times, in what was one of the strongest rebukes issued to data center development till that date.

The campaign started with emissions, and brought in a whole raft of complaints against the then little-known facilities. Some of the Times’ criticisms were off-beam, but that public pillorying appears to have stung Microsoft - its name will feature large in what follows.

No diesel at all?

What are the alternatives? For Microsoft, the first approach was to simply try to cut out the diesel. In what may have been a direct response to the Quincy campaign, Microsoft announced in 2012 that it would run some data centers with no diesel generators.

The company tried the idea in Boyton Virginia and Chicago, as well as Quincy. Microsoft’s David Gauthier, who was director of data center architecture and design management, said: “There are tens of megawatts in this data center and other data centers Microsoft runs. They do not have diesel generators behind them.”

At roughly the same time, in 2013, eBay set up a 6MW subsidiary data center in Salt Lake City using containers, and powered it with fuel cells as a primary source. The fuel cells ran from biogas, and the electricity grid took the place of the backup generators.

eBay’s data center architect at the time, Dean Nelson, said the change was worth it because it allowed for other changes, such as eliminating the hardware cost of the generators: “You can’t justify fuel cells without redesigning the core of the site,” he told this writer in 2013.

During that decade, there was optimism that multiple cloud data centers could use software-level redundancy to provide reliability. Loads would run in multiple sites, so a power outage in one place would not stop the software running.

In 2017, Andy Lawrence of Uptime Institute predicted that "over the next decade, for a sizeable but as yet undetermined number of operators, resiliency and redundancy at the individual data center level will, in whole or part, be complemented and/or replaced by resiliency at the IT level. This is not necessarily a resiliency tactic or strategy, but an inevitable move, resulting from the fact that applications themselves are becoming more distributed."

In five years we’ve not seen operators doing away with their UPSs, however.

eBay is understood to have switched to more conventional UPS approaches after Nelson left in 2016. Two years later, Microsoft also backtracked on its early enthusiasm for facilities with no generators. Effectively admitting failure, it issued a single request for 72 diesel generators at Quincy.

The idea won’t die

But in 2021, the concept re-emerged. After its 2020 commitment to be carbon negative by 2030, Microsoft needed a roadmap to eliminate its diesel consumption, along with other sources of emissions.

“Finding solutions to reduce reliance on traditional diesel fuels represents a substantial contribution to the technology pathways necessary for deep decarbonization,” the company said as it revisited the idea of changing its UPS approach.

This time round, it’s attacking the problem with a range of alternatives to petroleum diesel fuel.

“As a company, we're exploring all of the possible options,” says Microsoft senior program manager Casey Miller. “In the medium and long term, there’s different competing technologies - we're looking at technologies that may have an even lower carbon footprint.”

Let’s take a look at what those are.

Bio-methane microgrid

In San Jose, Microsoft is using renewable natural gas instead of diesel to power its backup. Enchanted Rock is installing a 100MW microgrid at an Azure data center being built on a site Microsoft bought in 2017, with 224 natural gas generators, each with a 0.45 MW capacity

The generators are fed by a natural gas supply. Enchanted Rock will source net-zero biogas captured from rotting food waste and agricultural operations, and inject it into the pipes upstream of the facility to offset any fossil gas burnt by Microsoft in the event of an outage.

As well as reducing greenhouse gases, the project helps the facility meet California’s tough clean air rules. Californian Democrat Senator Ben Hueso said: “I am hopeful that one day all data centers will replace their backup power systems with carbon-negative, clean renewable natural gas. Today’s announcement will set a precedent I believe all of California’s roughly 240 data centers should follow.”

Fuel cells and hydrogen

Fuel cells are a strong contender to replace diesel generators in the back up of data centers. They can be fed by various gases, including methane from natural gas or biogas, or hydrogen, which can be produced without emissions.

The fuel cell effectively ”burns” the gas fuel, combining it chemically with oxygen. If methane is used, then carbon dioxide is released into the atmosphere, which counts as green if the methane comes from a recent biological source.

If hydrogen is used, the only byproduct is water.

Microsoft’s efforts with fuel cells go back to 2013, when the company tested solid oxide fuel cells (SOFCs). These run at high efficiency, and can deliver a high load continuously - but as they don’t respond well to variable loads, they turned out not to be the best solution for backup systems which have to start quickly.

By 2018, the company switched to proton exchange membrane (PEM) fuel cells which can start up quickly as required by backup applications.

In 2020, Microsoft demonstrated a 250kW PEM system which provided continuous power for 48 hours to a bank of 10 racks of servers at a Microsoft energy lab in Seattle.

For two years, led by principal engineer Mark Monroe, the company ramped that up with the goal of creating a complete fuel cell system that could be plug-compatible with a diesel generator.

In 2022, it showed a 3MW power generation system powered by hydrogen, built by its partner Plug Power, and operating at Plug’s Latham, New York headquarters.

Plug created some larger (125kW) fuel cells, which enabled the project to fit into a unit the size of two 40ft shipping container, but - as a prototype - it is a lot more expensive than ordering a diesel generator. Still, the system passed the acceptance tests for a data center backup system, powering a simulated data center load (a set of heaters or “load banks”) during a full break in the grid power.

Microsoft’s director of data center research Sean James described this as “a moon landing moment. We have a generator that produces no emissions. It’s mind-blowing.”

The next step is to productize the prototype. Plug plans to make a smaller, commercial-grade version. Microsoft will test this at a real data center - possibly in an urban area where diesel generators are forbidden.

This test will be as much about the practicality of storing and using hydrogen, which is as big an issue as the fuel cells themselves.

Hydrogen fuel cells are only as green as the fuel they consume, and hydrogen can be a variety of colors.

“Green” hydrogen is produced by electrolyzing water using renewable electricity. That’s not widely available yet, so Microsoft is feeding its prototype with hydrogen produced as a byproduct in the industrial production of chlorine and sodium hydroxide.

Microsoft describes this as "blue" hydrogen - hydrogen which produces some emissions, but can be cleaned up.

So Microsoft wants to bring into being a hydrogen supply chain, and fuel cell backup units it can buy in quantity.

“We’ve got a commitment to be completely diesel-free, and that supply chain has got to be robust,” says James. “We’ve got to talk about scale across the entire hydrogen industry.”

The vegetarian option

A more basic and simple way to reduce diesel emissions, is to replace the fuel, not the generators. Diesels can burn vegetable oils, which release greenhouse gases that have only just been captured by plants, making them a "renewable" resource.

Petroleum by comparison, release greenhouse gases stored millions of years ago, which should left in the ground to avert the climate crisis.

First generation plant-based fuels didn’t do well, however. Biodiesel (EN14214) is produced from plants such as rapeseed grown for the purpose

“Biodiesels were just a nightmare,” comments Lex Coors, CTO at Interxion’s Digital Realty. “They were clogging up the systems, we need fuel polishers, and we needed, to replace the fuel every two or three years, regardless of the fact that we had all the polishers in place.”

Engine manufacturers hated biodiesels. “We even had to prohibit the usage of this kind of fuel with our engines, because it was really a disaster,” says Jacky Pluchon, vice president of EMEA power solutions at engine maker Kohler. “I mean, those engines were not built for that kind of fuel.”

By contrast, HVO (hydrotreated vegetable oil - EN15940) is a second generation fuel, produced from raw materials such as residues and waste.

Surprisingly, HVO scores a double whammy. It’s not just made from a waste product, making it part of a circular economy, it also can be used directly in existing diesel generators.

Most engine manufacturers are in the process of certifying their engines to burn HVO and say there’s no performance hit. In fact, performance improves.

“We’ve tested a lot of our generators, and we continue to do it, when they are sold to our customers, and the result is really good,” says Pluchon. “In fact, this kind of synthetic fuel is even better for the engine, because it doesn't create any impurities inside the engine. Technically speaking, it's really a great solution.”

He adds: “It was a very good surprise to us when we tested those generators, because we've tested other types of biofuel in the past, and the performance wasn’t the same.”

It’s also more stable, and easier to store than petroleum diesel fuel, and can be mixed with regular fuel: “You can keep HVO fuel five or six years,” says Pluchon. “That’s not the case with regular diesel. You need to have a fuel polishing system inside your tanks, or you need to change, change it on a regular basis.”

Data center operators like Coors’ Digital Realty want to use HVO100 (100 percent HVO) as a sustainable fuel: “In an emergency, if there is no HVO100 available, you can mix it to any percentage with normal diesel fuel.”

Coors backs up Pluchon’s performance claims: “The capacities are the same, we have no loss of capacity. The only disadvantage is the cost.”

HVO costs around double the price of conventional diesel, but Coors shrugs that off: “The time that we will need to earn the diesel is maybe an hour a year. That will not hurt our opex at all.”

Large batteries

Another option is to restructure the whole UPS system, with much larger batteries, which can support the data center long term.

There’s an obvious drawback to this approach, as batteries only store energy, they don’t convert it from a portable form like diesel generators do.

If an outage goes on for longer than a few hours, as can happen with incidents like floods and wildfires, then data centers can potentially carry on for long periods on no grid power, if they can replenish their tanks of diesel.

You can only replace a generator with a large battery if you are pretty sure no outage will last longer than the lifetime of the energy stored in that battery.

Despite this, Google is going ahead with replacing some diesel capacity with batteries, at its St. Ghislain, Belgium data center.

The search giant is replacing a diesel generator with a battery that can deliver 2.75MW of power, and hold 5.5MWh of energy. The positive benefit is that this battery can be connected bidirectionally to the grid, and support the grid when necessary.

The installation uses Gridstack batteries from Fluence, and is being connected up by Centrica, using Centrica's FlexPond software to store and discharge energy, so it can support the data center during power cuts and blackouts, and also provide up to half of the stored energy to the grid on a short term basis, to improve grid stability.

The batteries use LFP (lithium iron phosphate) batteries made by Chinese battery maker Contemporary Amperex Technology Co. This has a lower power density than more usual Li-ion chemistries, but has a better duty cycle and safety record, while using less toxic raw materials.

The installation only backs up a small proportion of the power needs at a single one of Google’s many facilities, but could be an important proof of concept.

Meanwhile, Microsoft is using Saft batteries in some UPSs, for a test being carried out with a partner which is evidently desperate for environmental credibility - the oil company Total. Needless to say, the emissions saved in this test will be utterly meaningless compared with those produced by Total as a whole.

Less testing

It’s worth mentioning there’s another way to reduce emissions from diesels, and that is to run them less.

Considering that data centers mostly switch their generators on only during tests, it makes a big difference if they are tested less.

Kohler is moving towards diesels that can be trusted without fully firing them up in tests, and it’s doing it because of pressure from the data center industry.

“It will require a different maintenance from the way it is done traditionally, but it's how we can improve - and I think everybody will follow that direction,” says Pluchon. “Data centers was the industry pushing for that. It didn't come from any other industry at this point.

Coors likes the prospect: “Kohler is one of the first companies who wholeheartedly supported my suggestion to go to zero run hours on diesel. We now see other companies also starting to change, because that's the only way forward. If you don't need a diesel, don't start it. Just start it when you need it.”

Positive spin-off

All these options are more expensive than diesel generators, or less available. In the case of hydrogen, they need a whole new production and distribution infrastructure.

But advocates argue that the data sector has high margins, and can afford to use more expensive options. In fact, it has a responsibility to do so, because it will stimulate a growing market for them.

Speaking of HVO, Miller says: “A potential influence we can have, as an industry, is to push legislation in that direction and bring more attention to adoption. Having an actual renewable diesel standard would go a big way. That's probably step one, making sure that there's a target that you have to hit with whatever you're designing your generators too.”

“We create the demand, and that will be followed by larger industries,” says Coors.

Of course, that raises a question. If demand ramps up, is there really enough fatty waste feedstock to support an expanding industry based on HVO?

“The source of the waste can be a question,” admits Pluchon. “There is still research done as to whether there are sources such as algae in the future. For now though, we produce a lot of waste on this planet, unfortunately. With the kind of resources there are, you will have a lot of it for the coming years.”

Even the long-term bets, like hydrogen, could pay off as well.

If data centers stimulate a market for distributing hydrogen, then a subsidiary market has to evolve for very practical reasons that Monroe pointed out in a DCD podcast. The small size of hydrogen molecules means there’s inevitable gradual discharge, so it can’t be stored for ever.

But that could be a positive spin-off. Hydrogen fuel cells would be a good power source for automotives, but there’s never been an available distributed source of green hydrogen.

The hydrogen slowly leaking from data center energy storage plants could be offered as fuel for cars powered by hydrogen fuel cells, and a full hydrogen economy could evolve.

"Once green hydrogen is available and economically viable, this type of stationary backup power could be implemented across industries, from data centers to commercial buildings and hospitals," says Monroe.