H-Storm: A Hybrid CPU-FPGA Architecture to Accelerate Apache Storm

The era of big data has led to the exponential growth of the amount of real-time data. Nowadays, traditional centralized solutions and parallelism techniques in distributed systems cannot satisfy the processing requirements of emerging applications. To overcome this inability, distributed stream processing (DSP) frameworks have emerged to utilize parallelism techniques and facilitate large-scale real-time data analytics. However, they are becoming impractical due to low throughput processing and inefficient resource utilization. In this paper, we design and implement a hybrid CPU-FPGA architecture based on Apache Storm (H-Storm), to improve processing throughput and average tuple processing time. H-Storm harnesses the computing power of FPGA by providing easy-to-use interfaces while preserving all strengths of Apache Storm. To utilize the FPGA resources, our architecture supports multiple accelerator interfaces to accelerate different tasks, simultaneously. An extensive evaluation of two different applications named Matrix Multiplication and Edge Detection shows that H-Storm can gain throughput improvement over the original Storm. To have a fair comparison, we used jBlas and OpenCV libraries as the rivals in full software implementations and the F-Storm framework in the hardware-accelerated implementation. Experimental results show that H-Storm archives up to 3.2X throughput gain and 2.3X speedup for Matrix Multiplication. It also leads to 3.4X throughput gain and 2.2X speedup for the Edge Detection application. Furthermore, several experiments are designed to determine when it is beneficial to use FPGA to accelerate compute-intensive components of the streaming applications.