Solving Millions of Eigenvectors in Large-Scale Quantum-Many-Body-Theory Computations

We present large-scale simulations of photovoltaics materials in the Vienna Ab initio Simulation Package [1] (VASP) that are only possible by pushing the boundaries of practically solvable eigenproblems. Thus, we enable analyses for semiconductors that were inaccessible within state-of-the-art predictive methods before. To achieve this, we a) implemented a distributed eigensolver in cuSOLVERMp that fully runs on the GPU, b) adapted VASP's Bethe-Salpeter Equation (BSE) algorithm for GPUs and to employ cuSOLVERMp, and c) dramatically improved the BSE workload distribution to yield near perfect load-balancing at scale. With a size of two million, we solve one of the largest dense, complex eigenproblems reported so far in under 5 hours sustaining 7.8 PFLOPS using 34.5 GJ on 4096 GPUs of NVIDIA's Selene supercomputer. Our results facilitate new breakthroughs in material science. These improvements in compute and energy efficiency apply to other domains relying on solving large eigenproblems, as well.