Among the list of NVIDIA announcements tonight at Computex 2022, the company revealed that it is preparing to launch liquid-cooled versions of its high-end PCIe acceleration cards. Offered as an alternative to traditional two-slot air-cooled cards, liquid-cooled cards come in a more compact single-slot form factor for improved cooling and improved density. The A100 liquid coolant will be available in the third quarter, and the H100 liquid coolant will be available early next year.
While liquid cooling isn’t new to a data center, it’s usually reserved for more dedicated hardware with extreme cooling and/or density requirements, such as the next generation of high-end NVIDIA H100 (SMX) servers. By contrast, PCIe servers are all about standardization and compatibility. For server video cards/accelerators, that means dual-slot cards designed for use with forced air cooling inside the server chassis. This serves the market segment well, but 300 to 350 watts of these cards mean they can’t get any thinner and still be effectively cooled by air – which in turn creates a 4-card limit for standard racking systems.
But times are changing, and liquid cooling is being implemented in data centers in larger capacities to keep up with cooling of ever-heating devices, and to improve the overall energy efficiency of data centers. To this end, NVIDIA will release liquid-cooled versions of the A100 and H100 PCIe cards in order to give data center customers an easy, officially supported path to install liquid-cooled PCIe accelerators within their facilities.
The cards (pictured above) are essentially a reference A100/H100 with the dual slot heatsink replaced with a single slot heatsink with a full coverage water block. Designed to be integrated by server vendors, it uses an open loop design that is intended to be used as part of a larger liquid cooling setup.
But other than changing the cooling system, the specifications of the cards remain unchanged. NVIDIA doesn’t increase the TDPs or clock speeds on these cards, so their performance should be the same as conventional air-cooled cards (as long as they’re not heat choked, of course). In other words, these new cards use liquid cooling to improve energy efficiency and density, rather than performance.
The first card out of the gate will be the liquid-cooled version of the 80GB A100 PCIe accelerator. It will be available to customers in the third quarter of this year. Meanwhile, a liquid-cooled version of the H100 PCIe is in development, and NVIDIA expects that to be available in early 2023.
In the meantime, NVIDIA is working with Equinix to qualify the liquid-cooled A100 within their data centers, as well as to get an idea of the real-world energy savings for the new hardware. Interestingly, NVIDIA reports a significant reduction in overall data center power usage from switching to liquid cooling—the 2000 server setup (4,000 A100 card) saw an overall reduction in power needs of 28%. According to NVIDIA, this is the result of a combination of overall power savings across the data center from the switch, including everything from improved video card energy efficiency from lower temperatures to lower power needs from cooling water versus powering large air coolers. All of this underscores why NVIDIA is promoting liquid-cooled devices as an energy efficiency gain for data center operators looking to reduce energy use.
And while this first generation of liquid-cooled devices focus on efficiency, according to NVIDIA, that won’t always be the case. For future generations of cards, the company will also consider liquid cooling to improve performance at current power levels — likely by investing data center scale gains back into the cards’ higher TDPs.
Finally, while the bulk of NVIDIA’s announcement today (in addition to the case study) focuses on PCIe cards, NVIDIA is also revealing that it’s working on official liquid-cooled designs for its HGX systems as well, which are used in the company’s more powerful SMX cards. The liquid-cooled HGX A100 is already shipping, and the liquid-cooled HGX H100 is scheduled for release in the fourth quarter.