Dynamic Fine-Grained Workload Partitioning for Irregular Applications on Discrete CPU-GPU Systems

As the scale of big data continues to grow and the complexity of data analysis algorithms increases, the desire for greater computing power is increasingly evident. A popular approach is to utilize heterogeneous systems for computation. Discrete CPU-GPU system, which has CPU and GPU on different chips connected through a PCI-e bus, is a typical heterogeneous system. There have been a lot of schemes proposed to improve performance on the discrete CPU-GPU system through partitioning. However, most of them are for regular applications and far from ideal in terms of resource utilization and performance for irregular applications. Heterogeneous computing receives much attention due to its performance potential, and many regular applications are greatly benefited from it, but the acceleration of irregular applications is still a problem to be solved. In this paper, we propose a dynamic fine-grained workload partitioning approach for irregular applications that boosts resource utilization to achieve a better load balance on heterogeneous platforms. The approach monitors the kernel execution of the CPU and GPU at runtime, and finely partitions the workload according to their processing speed, assigning relatively regular data to the GPU and the rest to the CPU. Evaluated with various irregular workloads, our scheme achieves up to 20% performance improvement over the state-of-the-art coarse-grained scheme, and the performance gap is less than 5% in most cases compared to oracle-based partitioning.