Distributed In-Network Coflow Scheduling

Recently, there has been a growing interest in coflow scheduling due to the rise of data-intensive applications. However, existing solutions rely on modifying hosts to obtain coflow information and cooperatively prioritize their packets. Such a host-assisted approach may not work for public data centers and could be problematic when the central controller becomes a bottleneck. In this work, we present PICO, an in-network coflow scheduling system allowing a programmable switch to prioritize coflows in a fully distributed way. In the absence of host cooperation, we develop a pairwise coflow detection scheme that clusters sequentially arrived flows. We further design a data plane pipeline that enables fast feature extraction and efficient coflow size tracking for real-time priority adaptation. The experiments show that our sequential coflow grouping achieves an accuracy of up to 99%. The coflows, on average, complete 1.28× faster than per-flow fair sharing, showing the effectiveness of PICO's distributed in-network scheduling even with no hardware modification and host cooperation.