ONCache: A Cache-Based Low-Overhead Container Overlay Network

Recent years have witnessed a widespread adoption of containers. While containers simplify and accelerate application development, existing container network technologies either incur significant overhead, which hurts performance for distributed applications, or lose flexibility or compatibility, which hinders the widespread deployment in production. We design and implement ONCache (\textbf{O}verlay \textbf{N}etwork \textbf{Cache}), a cache-based container overlay network, to eliminate the overhead while keeping flexibility and compatibility. We carefully analyze the difference between an overlay network and a host network, and find that an overlay network incurs extra packet processing, including encapsulating, intra-host routing, namespace traversing and packet filtering. Fortunately, the extra processing exhibits an \emph{invariance property}, e.g., most packets of the same flow have the same processing results. This property motivates us to cache the extra processing results. With the proposed cache, ONCache significantly reduces the extra overhead while maintaining the same flexibility and compatibility as standard overlay networks. We implement ONCache using eBPF with only 524 lines of code, and deploy ONCache as a plugin of Antrea. With ONCache, container communication achieves similar performance as host communication. Compared to the standard overlay network, ONCache improves the throughput and request-response transaction rate by 12\% and 36\% for TCP (20\% and 34\% for UDP), while significant reduces per-packet CPU overhead. Many distributed applications also benefit from ONCache.