Task-based low-rank hybrid parallel Cholesky factorization for distributed memory environment

The primary targets for improving efficiency for large-scale matrix factorization are reducing synchronization, addressing the overlap in communication and computation, and improving load balance. In recent years, tiled algorithms with task parallelism in multicore shared memory systems have become well-established as efficient methods for conducting fine-grained computations on smaller tiles. Moreover, they provide flexible execution orders for a runtime system in many situations. However, traditional hybrid programs with MPI and OpenMP for distributed memory systems use a fork-join model for multi-threads in each process, which leads to thread-parallel computing tasks interchange with sequential communication tasks. In this paper, we incorporate task parallelism and low-rank approximation into a hybrid task-based Cholesky factorization in a distributed environment and propose some low-rank variants. We evaluate the performance of our programs on both full-rank inputs and low-rank inputs and report the pros and cons of the proposed programs.