Beehive: A Flexible Network Stack for Direct-Attached Accelerators

Accelerators have become increasingly popular in datacenters due to their cost, performance, and energy benefits. Direct-attached accelerators, where the network stack is implemented in hardware and network traffic bypasses the main CPU, can further enhance these benefits. However, modern datacenter software network stacks are complex, with interleaved protocol layers, network management functions, as well as virtualization support. They also need to flexibly interpose new layers to support new use cases. By contrast, most hardware network stacks only support basic protocol compatibility and are often difficult to extend due to using fixed processing pipelines. This paper proposes Beehive, a new, open-source hardware network stack for direct-attached FPGA accelerators designed to enable flexible and adaptive construction of complex protocol functionality. Our approach is based on a network-on-chip (NoC) substrate, automated tooling for the independent scale-up of protocol elements, compiletime deadlock analysis, and a flexible diagnostics and control plane. Our implementation interoperates with standard Linux TCP and UDP clients, allowing existing RPC clients to interface with the accelerator. We use three applications to illustrate the advantages of our approach: a throughputoriented erasure coding application, an accelerator for distributed consensus operations that reduces the latency and energy cost of linearizability, and TCP live migration support for dynamic server consolidation.