Computers submerged in fluid may reshape data-center cooling design

Water and power increases for cooling no longer inevitable.

COOLING OFF: David Smith, front, and Dave Martinez, behind, work on a computer server submerged in liquid as part of testing at the High Performance Computing center at Sandia. The immersion system uses a nonconductive liquid coolant that enables 100% of the heat generated to be captured. The reflection of the computer demonstrates the high liquid level. [Credit: Photo by Craig Fritz]

 

 

 

 

By Neal Singer, Sandia

Keeping electrical circuits dry is generally considered a vital safety measure, but at Sandia's High Performance Computing center, technicians adjust live circuits submerged in liquid.

"It's pretty cool -- and a little unusual -- to walk into a computing lab and see electrical hardware submerged in fluid," Sandia engineer David Damm said.

Complete immersion of computer components appears to be the most effective way to cool them, said Dave Martinez, engineering program project lead for Sandia's Infrastructure Computing Services. "We place whole computers -- their power cables, everything -- in a liquefied solution. We take an entire rack and drop it into fluid contained in big immersion tanks."

He believes that this direct contact between a liquid coolant and electrified equipment "could reshape the future designs of data centers."

Reshaping data-center architectures
By submerging all parts of the computing servers in a liquid coolant that doesn't conduct electricity, 100% of the generated heat can be captured, almost entirely eliminating the need for the power-hungry fans and chillers used in conventional cooling systems.

Martinez estimates that this type of cooling system, using liquid approximately the viscosity of cooking oil, would cut energy consumption by 70%. Unlike water-chill systems that require evaporation to lower operating temperatures, no water is lost; the coolant gives up its heat to the open air, given the right temperature differential.

VIDEO: COLD PLUNGE: David Smith makes network connections to a computer server submerged in liquid as part of testing at the High Performance Computing center at Sandia. [Credit: Video by Craig Fritz]

 

 

Sandia's forward-thinking approach arises from projections that the increasing water and electrical power demands for cooling in high-performance computing eventually will exceed the resources of small towns and become an unsupportable burden.

To address this, Sandia is formally testing a commercial nonconductive liquid system from Barcelona-based company Submer Technologies throughout the summer. Early positive results led Martinez to envision a reduction in the two big problem areas of power and water use that are expected to hinder the development of faster, more powerful machines.

Promising innovations and partnerships
According to the Submer website, their single-phase immersion cooling system uses a biodegradable, nontoxic, nonflammable and noncorrosive liquid. It boasts being 95% more efficient than traditional cooling technologies, while 1,400 times a better thermal conductor than air and eight times less electrically conductive.

Negotiating through Adacen Inc., a local Albuquerque data company, Martinez convinced Submer to partner with Sandia. Submer would provide fluid and equipment, and Sandia would provide its expertise in system cooling and testing to perform a thorough technical evaluation of the system. Meanwhile, Martinez, who for 25 years has worked diligently to bring about the next stages of computer cooling, is happy to reduce Sandia's costs for power and water and provide a stage for a potential revolution for the industry.

In partnership with Adacen, code is being written specifically for the new task by Hoonify Technologies, composed of five former Sandia researchers who have taken entrepreneurial leave from the Labs.

TEMPERATURE DROP: Hoan Luu, left, and David Damm work on a computer server submerged in liquid. Save for the silver handles at the top, the entire computer server is submerged. [Credit: Photo by Craig Fritz]

 

 

 

 

In 2020, Submer, in collaboration with Intel Corp., introduced the idea of a liquid coolant that would bathe all components of a computing system to provide even cooling and allow higher operating temperatures.

As a bonus, the warmer temperature of the returning liquid can offer some heat to adjacent buildings during winter months and can be used to heat labs and showers in the summer through heat-exchange processes.

Sandia will conduct comprehensive tests on this system, evaluating hardware reliability and infrastructure support cost and effort. A case study is expected to be released this fall to assess the pros and cons of the technology.

Designfax Editor's Note: We asked author Neal Singer if the computer parts in this research were coated/encapsulated with anything special, or if they were standard, off-the-rack parts/computer systems. He contacted Dave Martinez, project lead, who said, "There is no special coating on the computer; the power cables have an added part on the ends to keep the liquid from traveling outside the tank."

Read more Sandia Lab News articles at sandia.gov/labnews/.

Sandia National Laboratories is a multimission laboratory operated by National Technology and Engineering Solutions of Sandia LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration.

Published June 2024

Rate this article