Aquila: A Distributed and Portable Post-Processing Library for Large-Scale Computational Fluid Dynamics

High-speed, compressible, turbulent boundary layers will be continue to gain relevance in many military, space and future civilian applications. However, furthering our understanding of crucial aspects of such flows present complex computational challenges from the preparation to the execution of said simulations. For instance, a relatively modest DNS could generate 12-billion-data points with a simple geometry. This can be attributed in part to the resolution required to capture the smallest scales of motion at higher Reynolds numbers. More complex geometries could generate 16 TB of raw data. If more complex geometries are to be explored at higher Reynolds numbers, the need for scalable post-processing utilities will continue to increase at an exponential pace. Furthermore, present times have seen the advent of increasingly more heterogeneous and diversified computing infrastructure which adds significant complexity to the development of a portable solution. Consequently, we propose Aquila, a performance-portable, distributed library capable of handling the post-processing of large-scale simulations while alleviating the burden from the domain expert through a modern abstraction mechanism that hides unnecessary complexity from the application writer. Aquila builds over two major abstractions a distributed flow field and modular operations. The distributed flow field leverages both Kokkos and any distributed computing library wrapped around the Aquila Communicator interface. Kokkos enables Aquila to execute transparently and with performance portability on both CPUs and GPUs from multiple vendors while abstracting the distributed computing layer enables the adoption of virtually any distributed memory communication library although MPI is the only currently supported backend. Aquila is portable from laptop computers to supercomputers with no changes to the source code. In this paper, we present Aquila's design rationale and implementation details. A key distinction in Aquila's design is the scalable inclusion of out-of-core post-processing pipelines with data pre-fetch to give the illusion of in-memory availability of files. Furthermore, we show preliminary results, from a performance and usability point of view, on porting a post-processing analysis to Aquila.