AutoConstruct: Automated Neural Surrogate Model Building and Deployment for HPC Applications

Scientific Machine Learning (SciML), aiming at using machine learning methods to solve scientific computing problems, has been used in a wide range of HPC applications to improve the applications' performance. However, domain scientists, despite their rich expertise in their respective fields, often lack adequate knowledge of machine learning and computer systems. This gap makes it difficult for them to determine which computations can be replaced by neural network (NN) surrogates. Moreover, NN surrogates in HPC applications typically have sparse input features, necessitating system-level optimization to prevent memory resource overuse on accelerators. Consequently, constructing and deploying NN surrogates for HPC applications remains a significant challenge, with no existing tools to assist domain scientists in systematically building high-quality ML surrogates. To address these issues, we introduce AutoConstruct, a comprehensive solution for SciML in distributed systems. AutoConstruct consists of two components, the model-maker and deployment-helper, which operate automatically to select the appropriate ML surrogate architecture and deploy the surrogate model training and inference on specific accelerators, respectively. We showcase AutoConstruct's adaptability and performance through a series of HPC numerical benchmarks, real-world scientific applications, and computing platforms.