Carpe Elephants: Seize the Global Heavy Hitters

Detecting "heavy hitter" flows is the core of many network security applications. While past work shows how to measure heavy hitters on a single switch, network operators often need to identify network-wide heavy hitters on a small timescale to react quickly to distributed attacks. Detecting network-wide heavy hitters efficiently requires striking a careful balance between the memory and processing resources required on each switch and the network-wide coordination protocol. We present Carpe, a distributed system for detecting network-wide heavy hitters with high accuracy under communication and state constraints. Our solution combines probabilistic counting techniques on the switches with probabilistic reporting to a central coordinator. Based on these reports, the coordinator adapts the reporting threshold and probability at each switch to the spatial locality of the flows. Simulations using traffic traces show that our prototype can detect network-wide heavy hitters with 97% accuracy, while reducing the communication overhead by 17% and switch state by 38%, compared to existing approaches.