Developing new and better liquid coolants for immersion cooling is not as easy as it might seem. They must be non-corrosive and non-toxic so that equipment immersed in the coolant is safe enough to handle. They must also offer high thermal conductivity and low viscosity. And, of course, they have to have a high flashpoint, not just as a liquid but also, potentially, in any vapor form.
Even better would be a completely stable coolant that does not evaporate at all.
As a result, conventional immersion cooling has been based on mineral oil because it is stable, highly resilient to oxidation and therefore reasonably long lasting. But quite apart from the handling issues, over time, mineral oil will nevertheless change color and start to develop a characteristic foul odor.
And that’s not all. The PVC cables plugged into the server blades can stiffen, making them harder to handle and, more seriously, the servers’ capacitors can swell, requiring regular, close inspections. Some interface materials, furthermore, can go soft in mineral oils.
That’s why Green Revolution Cooling (GRC) developed its line of ElectroSafe® coolants. Not just as an alternative for the liquid coolant long used for mainframes and supercomputers but, for the first time, as a coolant suitable for mainstream use with standard servers in data centers.
ElectroSafe coolants were introduced in 2014 and are used in GRC’s ICEraQ® and ICEtank® modular immersion cooling systems. The liquid itself has its own guarantee: to maintain cooling capacity for the lifetime of the data center – that is to say, 15 years or more – without any of the kind of degradation associated with mineral oils.
“Most important of all, ElectroSafe provides better cooling performance, as well as improved viscosity over mineral oil,” says Larry Kosch, director of product marketing at GRC.
He continues: “Mineral oil will last a long time, but it starts to turn a different color and develop an odor which, if you’re a user is going to make you wonder whether it’s good or starting to go ‘off’ and needing to be replaced.”
As single-phase coolants, ElectroSafe maintains an entirely liquid state; they do not evaporate and, therefore, do not require the hermetically sealed environment in which to operate that two-phase coolants need in order to be used safely. Indeed, there isn’t even a requirement to use maintenance gloves when installing or servicing servers immersed with ElectroSafe coolants.
Fit for purpose
ElectroSafe is an umbrella term for GRC’s range of synthetic liquid coolants. One coolant is generally not suitable for all immersion applications, whether due to environmental, regulatory, supply availability, cost, or other customer considerations.
“Having a broad spectrum of options available enables GRC to deliver the best immersion cooling solution that suits all of our clients’ individual requirements, while remaining safe for the environment and enabling the low total cost of ownership that the adoption of single-phase liquid immersion cooling offers,” says Kosch.
Liquid cooling is a relative newcomer to the data center. While adopted for mainframes and supercomputers as far back as the 1980s in order to help push the limits of performance, it is increasingly being used in more mainstream computer architectures for the same reason.
In addition, though, it also offers a number of environmental advantages: liquid cooling enables servers to be run more efficiently, enabling more compute power to be supported in the same space, and the dissipated heat is more easily captured for the purpose of re-use, such as heating greenhouses, fish farms or for use in district heating schemes.
Furthermore, it also makes for a quieter and more clement workspace in data center halls – air conditioning doesn’t need to be maxed out just to prevent server meltdown.
GRC’s ElectroSafe family is based on synthetic hydrocarbon-based coolants, with 1,200 times the capacity for absorbing heat than air. In addition to being successfully used by GRC customers over the past eight years, ElectroSafe has also been thoroughly tested by key technology partners such as Dell, Intel, and Vertiv.
Immersion cooling is compatible with any OEM server, notwithstanding a few straightforward tweaks to take account of the different method of cooling. That includes an adjustment to the server bios to take account of the removal of fans from the motherboard, and the swapping of standard PVC-jacketed cabling with plenum-rated cables.
ElectroSafe coolants also offer a high flashpoint, with options compliant with the exceptionally high standards set by authorities in Japan and Taiwan of 250 degrees Celsius.
In addition to being inert and non-evaporative, it contains no compounds that could contribute to global warming and is both recyclable and biodegradable, as well as non-toxic.
Furthermore, immersion system maintenance requirements are not onerous: they pretty much amount to just a filter change on GRC’s ICEraQ immersion coolers once a year, with no downtime. Furthermore, as ElectroSafe coolants don’t evaporate, they don’t even need periodic top-ups, either.
The ICEraQ commeth — again
ElectroSafe is used in all of GRC’s ICEraQ modular immersion cooling systems, including the latest Series 10, a completely new line designed from the ground up. It is, says Ben Smith, chief product officer at GRC, is “a real re-imagination of everything we've done before, really taking into account the lessons learned over 12 years of product installations and service with our customers.”
The primary advantage of the Series 10, says Smith, is that it is completely integrated: Earlier coolant distribution units (CDUs) needed space for performing maintenance, piping and other elements that formed part of their modular design. Both the space required and the way in which the different parts fitted together could be off putting to prospective customers, admits Smith.
“We built a completely integrated system where the coolant distribution, the racks, the containment, and the distribution module are easily integrated into a single system at the customer site,” Smith stated during a DCD Tech Innovation Day after the product had been launched.
The ICEraQ Series 10 CDU occupies just 40 percent of the footprint of its predecessor, enabling more super-cooled server power to be concentrated on less real estate. But its more compact size does not come at the expense of accessibility: anything that requires direct maintenance of any kind is located at the front, with completely redundant pumps and heat exchangers to keep mission-critical loads going, hitch-free.
An indication of how seriously immersion cooling is now being taken is the recent collaboration agreement signed between GRC and Intel, deepening the relationship between the two companies for developing and implementing advanced immersion cooling technology for data centers, a relationship that dates back to 2012 when Intel conducted a year-long test.
In that investigation, Intel concluded that immersion cooling could slash data center PUE from an average, then, of 1.6 in typical air-cooled environments, to just 1.02 to 1.03.
Under their new agreement, Intel and GRC will research and develop new, even better coolant fluid formulations, testing them to make sure that they meet stringent standards for safety, material compatibility, and thermal performance. They will also work to optimize GRC immersion-cooled racks for Intel Xeon processors. Finally, the two companies will work together with joint end-customers to align and validate their solutions.
In addition, GRC also has close partnerships with server giant Dell and data center infrastructure provider Vertiv, as well as high-end server maker Supermicro. With Dell, GRC is setting up global SKUs for complete solutions – offerings that bundle server packages together with services. Vertiv, meanwhile, provides GRC as an immersion cooling solution for its large data center clients.
Technology giants like Dell and Intel are investing in immersion cooling for a good reason: customer and regulatory pressures around sustainability are only going to intensify, and immersion cooling enables data center operators to maximise server performance, getting more for less. Not only that, but the ability to squeeze more compute power into smaller spaces can save money, especially in cities where suitable real estate is increasingly scarce and expensive.
With GRC’s ‘try before you buy’ program, you can verify immersion cooling’s performance before taking the plunge
We sat down with John Bean, CTO at GRC to talk about the sustainable data centers, and myths surrounding liquid cooling