This conference paper was submitted for presentation at the NAFEMS World Congress 2025, held in Salzburg, Austria from May 19–22, 2025.

Abstract

High-performance computing (HPC) continues to play a pivotal role in driving innovation across industries, from design and simulation to AI-enabled engineering. As workloads increasingly shift between on-premises and cloud environments, orchestrating hybrid HPC systems has become essential to ensure performance, cost-efficiency, and scalability. This presentation will delve into strategies for addressing data gravity challenges and implementing robust data management solutions in hybrid HPC architectures. A central focus will be on the evolving role of data strategies in HPC environments. The discussion will cover the importance of tiered storage strategies, highlighting how balancing fast-access storage tiers with cost-effective archival solutions can optimise operational costs without sacrificing performance. Emphasis will be placed on the critical role of Simulation Process and Data Management (SPDM) and Simulation Process and Data Resource Management (SPDRM) systems in ensuring data provenance, reproducibility, and accessibility for downstream use cases. One of the key innovations to be discussed is the Data Lake Exporter (DLE) solution. This approach automates the transfer of HPC workloads executed in the cloud to archival cold storage, enabling organisations to minimise costs while preserving simulation results for future use. Furthermore, this process begins the transformation of raw simulation data into AI-ready datasets, opening pathways to leverage machine learning and AI for predictive modelling, optimisation, and insight generation. The presentation will also explore best practices for metadata capture to ensure simulation data is not only stored but also structured for effective retrieval and AI utilisation. Real-world use cases and insights from deploying hybrid HPC solutions across diverse industries will provide context for the strategies shared. Attendees will leave with a deeper understanding of how to design and manage hybrid HPC environments that seamlessly integrate compute, data management, and AI workflows while addressing sustainability goals and operational efficiency. This session will be particularly valuable for engineering leaders, data scientists, and HPC practitioners aiming to harness the full potential of hybrid architectures in their simulation and AI endeavours.