Benchmarking Summit and Sierra Supercomputers : From Proposal to Acceptance

The deployment of large-scale high-performance computer systems requires a thorough acceptance process to validate systems' specifications, including performance projections made using a predefined set of benchmarks at the time the system was proposed. In this paper, we describe such a process for the recently deployed Summit and Sierra supercomputers, starting from benchmarks projections made four years in advance to system availability, up to measurements prior to the system acceptance. This is an industry experience paper, reflecting on the “art of benchmarking” throughout the entire development of these supercomputers, highlighting the codesign process and the success in the benchmarks projections.