Nvidia’s direct-to-chip liquid cooling is a sustainability and a performance win | Tech Radar
Overview
News, deals, reviews, guides and more on the newest computing gadgets
Start exploring exclusive deals, expert advice and more
Details
Unlock and manage exclusive Techradar member rewards.
Unlock instant access to exclusive member features.
Get full access to premium articles, exclusive features and a growing list of member rewards.
Nvidia’s direct-to-chip liquid cooling is a sustainability and a performance win
Nvidia’s liquid cooling cuts water waste and boosts performance-per-watt
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Nvidia’s closed-loop liquid cooling system virtually eliminates water waste
Direct-to-chip cooling transfers heat more effectively than air
It enables higher performance-per-watt and higher rack densities
Data centers aren’t without their fair share of criticism – energy-intensive compute raises temperatures and giant campuses consume considerable amounts of air and/or water to keep them running optimally.
Land scarcity and financial incentives have also been pushing new developments closer to high-risk areas, including drought-prone regions, ultimately leading to even higher cooling requirements.
But Nvidia knows this, and it knows traditional air cooling has pretty much reached its limits as AI hardware becomes denser and denser.
Closed-loop cooling virtually eliminates water waste
With cooling now a core part of AI infrastructure design, Nvidia’s latest liquid-cooled AI systems promise higher thresholds to reduce the burden, driving down water and energy consumption as a result.
By running coolant at higher temperatures – 45°C or 113°F, to be specific – it enabled simpler cooling systems and lowers electricity consumption. Nvidia’s concept uses 75% water, 25% glycol as a coolant, noting that it can run about 5-7°C higher than hot tubs.
Cooling just became the most strategic choice in AI infrastructure
Saltwater cooling system could turn data centers into clean water producers
Nuclear engineer wants to use special bubbles to save AI data centers from a massive energy crisis
Compared with traditional evaporative cooling towers, Nvidia’s latest proposition involves a closed-loop system where coolant continuously circulates through servers to remove excess heat from chips. The warm coolant then cycles through external dry coolers leaving virtually no water evaporation.
The company boasted that cooling-related water consumption can be reduced by as much as 100% in suitable climates subject to the occasional extreme day, with cooling towers totally eliminated.
“The Nvidia DSX reference design for AI factories has zero water consumption – we have eliminated massive amounts of power usage and pretty much all water usage,” Data Center Cooling and Infrastructure Director Ali Heydari said.
The system’s efficiency primarily comes from direct-to-chip liquid cooling, where liquid flows directly through cold plates attached to CPUs and GPUs. This captures and expels the heat from exactly where it’s produced.
Not only is this more effective than cooling entire rooms, but liquid also promises to transfer heat thousands of times more effectively than air.
The AI heat trap: why data centers must rethink thermodynamics
Micron's $50 billion chip factory in Idaho will use billions of litres of water every year
Amazon reveals exactly how much water its data centers used last year
Major improvements across water consumption, energy efficiency and Power Usage Effectiveness (PUE) all help on the sustainability front, but there’s another layer to the benefits.
Nvidia says it can increase rack density and performance
Nvidia acknowledged that chip power consumption and rack densities continue to rise, so by implementing liquid cooling data center companies can add more GPUs per rack, use higher rack power and ultimately pack in larger AI clusters within the same building footprint.
The company explained that its Rubin systems now fit inside two racks, instead of six, marking a major space saving.
Simultaneously, air cooling has become ineffective. “Once the watts per chip crossed a certain level, liquid cooling became mandatory,” Motivair’s CEO Richard Whitmore said.
Independent testing totally separate from Nvidia’s latest announcement shows that its H100 systems delivered around 17% higher performance when water cooled, compared with air cooling. Under sustained AI workloads, GPU temperatures fluctuated between 41-50°C when water cooled, and 54-72°C when air cooled.
Besides improving immediate and sustained performance, greater thermal efficiency could also boost longevity.
The new, higher-temperature closed-loop water cooling model is set to be used in upcoming Rubin deployments this year.
Follow Tech Radar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.
With several years’ experience freelancing in tech and automotive circles, Craig’s specific interests lie in technology that is designed to better our lives, including AI and ML, productivity aids, and smart fitness. He is also passionate about cars and the decarbonisation of personal transportation. As an avid bargain-hunter, you can be sure that any deal Craig finds is top value!
You must confirm your public display name before commenting
1'They come in with an energy and excitement, a new view on things': AWS CEO Matt Garman explains why Amazon is hiring 11,000 new interns and junior staff when others are getting rid
2I tested the Leica SL3-P, and it's a masterclass in restrained design
3 Amazon's Prime Day 4K Blu-ray sale is heaven if you want movies in Dolby Atmos and HDR to really show off what your home theater can do — I've picked 20 of the best, all under $15 / £17
4'We're going to look back at this day as the moment we shifted safety into the next gear': Samsara's new 360 camera and AI tools look to make work sites safer and smarter for all
5 The i Pad Air is my favorite Apple tablet, but it just got more expensive — this Prime Day deal may be your last chance to grab it for less, and it’s already selling out
Tech Radar is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site.
© Future US, Inc. Full 7th Floor, 130 West 42nd Street, New York, NY 10036.
Key Takeaways
- News, deals, reviews, guides and more on the newest computing gadgets
- Start exploring exclusive deals, expert advice and more
- Unlock and manage exclusive Techradar member rewards
- Unlock instant access to exclusive member features
- Get full access to premium articles, exclusive features and a growing list of member rewards
