Remora : A MPI runtime for Composed Applications at Extreme Scale

As on-node computational power continues to outstrip I/O bandwidth HPC workflows are increasingly composed into communicating processes on the same node. While MPI provides methods of composition, such as MPMD jobs and intercommunicators, they require significant implementation overhead in the composed applications themselves, either via knowledge of the structure of the composition or via an intimate understanding of the mappings between applications. This paper describes a new MPI runtime, Remora, currently in progress, which uses the composition primitives of the Hobbes project to make it possible to compose MPI applications in such a way that minimizes implementation overhead. Remora requires minimal changes to the applications themselves, preserving traditional MPI COMM WORLD semantics while using intracommunicators instead intercommunicators in order to limit platform specific dependencies.