A Functional Approach for the Automatic Parallelization of Non-Associative Reductions via Semiring Algebra

Recent developments in computer architecture have yielded to a surge in the number of parallel devices, such as general purpose GPUs and application specific accelerators, making parallel programming more common than ever before.However, parallel programming poses many challenges to the programmer, as implementing correct solution requires expert knowledge of the target platform, in addition to advanced programming skills. A popular solution to this problem is the adoption of a pattern-based programming model, wherein the code is written using high-level transformations. While automatic parallelisation for some of these transformations is easily performed, this is not the case for others. In particular, the automatic parallelisation of Reduction is commonly supported only for a restricted class of well known associative operators: outside of these cases, the programmer must either resort to handwritten ad-hoc implementations or settle for using a sequential implementation. This document proposes a method for the automatic parallelisation of Reductions which, by taking advantage of properties of purely functional programs and the algebraic notion of a semiring, is capable of handling cases in which the reduction operator is not necessarily associative, but instead satisfies a much weaker set of properties. It then presents a series of additions to the Lift compiler implementing said methods, and finally proceeds to evaluate the performance of optimised Lift programs, demonstrating that the automatically parallelised code can outperform equivalent handwritten implementations, without requiring any explicit intervention from the programmer.