A Novel Approach to Combining 1D and 3D Simulations to Accurately Model the Immersion Cooling of CPUs and GPUs of a Blade Server

This presentation was made at the 2019 NAFEMS World Congress in Quebec Canada

Resource Abstract

In a modern data center, for every kilowatt of computing power that is deployed, there are potentially several hundred further watts of power required to support it. The majority of this energy is utilized in moving coolants around the data center. It is an energy intensive process that also loses a lot of energy along the way. One of the major advantages of liquid cooling is that liquids are generally manageable; liquids fill containers and move around them in a much more controllable and predictable fashion which means you don’t have to put as much energy into moving them around as you would have to for air.

In order to address the challenges of the data center market, an investigation was done to determine if current generation liquid cooling technology could be adapted for this market. This lead to the exploration of design changes such as 1U form factor, horizontal deployment, and backward compatibility for rack designs where immersive liquid cooling technology can have the most benefit.

This posed significant design challenges for the engineering team. All these challenges were met and overcome using CFD techniques that included a close coupling of 1D and 3D CFD tools that included Simcenter Flomaster and Simcenter FloEFD. This allowed for quick and accurate co-simulation of the entire system.

Combining 1D and 3D CFD simulation is not a new idea, BMW was creating coupled models between Flowmaster2 and Star-CD in the early 2000’s for designing water jackets for IC engines. [1] The approach they used was to create a direct coupling of the two software for co-simulation. While this was a valuable step forward, the coupling had to be done manually for each new system or component making it cumbersome to manage. Additionally, the long analysis times actually were an impedance to the systems analyses which would normally run in seconds for a steady state run now took hours to days using the coupled approach. There were definite advantages for the 3D simulation. It allowed for larger adjustments to the boundary conditions which in turn reduced the number of iterations required.

There are now commercial products available to aid with this coupling such as MPCCI from the Fraunhofer Institute which handles the functions of passing data from one tool to the other and manages iteration and time step synchronization. This is a benefit from the 3D CFD side because it can dramatically reduce the set up time for a single simulation but it does little to improve the performance of the 1D system simulation.

The approach used by Simcenter Flomaster and Simcenter FloEFD is different than past approached as it does not require a middleware program to manage the passing of data between programs and sequencing time steps. Rather, the tools share a common solver matrix and both models are solved simultaneously. This provides simulation efficiencies throughout the process by reducing the time to set up the co-simulation, reduced data handling and simpler post processing.

Document Details

AuthorCroegaert. M
Date 18th June 2019
OrganisationMentor Graphics Corporation


Back to Search Results