An Efficient Scheduler for Task-Parallel Interactive Applications

Modern software is often interactive - applications communicate frequently with the external world. For such applications, responsiveness - how quickly they respond to requests - is as important as throughput. Efficiently implementing these applications using traditional processes or static threads is difficult and error-prone. Task parallelism has the potential to significantly simplify the implementation of these applications - it allows the programmer to express the high-level logical flow of the program and letting the scheduler handle the low level details of scheduling, synchronization, and asynchronous I/O operations. Researchers have begun to study how to best support such interactive applications where different components have different responsiveness requirements on priority-oriented task-parallel platforms. We propose Prompt I-Cilk, a practically efficient scheduler for priority-oriented task-parallel interactive applications. Our scheduler exhibits superior performance when compared to the state-of-the-art scheduler design, including on the Memcached object server, a large scale real-world interactive applications that we ported to run on a task-parallel platform. Our scheduler design defies the conventional folk wisdom on how to schedule task-parallel code - we moved away from randomized work stealing, and we implemented "prompt" scheduling with frequent checking of core-to-priority-level assignments. We show that such design choices make sense based on the workload characteristics of the parallel interactive applications we tested.