OEM: An operation-aware event matching algorithm for content-based Pub/Sub systems

Due to the continuous expansion of the user base and the heightened demands for quality of service (QoS), real-time data distribution has encountered significant challenges in event matching within large-scale content-based publish/subscribe (CPS) systems. Many efficient algorithms have been proposed to enhance the matching performance. However, most existing algorithms scarcely consider operation characteristics, which often leads to performance degradation due to a multitude of repetitive operations, such as comparisons, additions, and assignments. In this paper, we propose an Operation-aware Event Matching algorithm called OEM, which takes into account the efficiency of CPU operations, SIMD instructions, and multi-thread architecture. First, we introduce a bitset-based subscription cache mechanism (SCM), which enables the efficient execution of bitwise OR operations during the matching process. Furthermore, we propose two optimizations, namely an optimal skewness-aware cell grouping (OSCG) strategy and an attribute-based clustering mechanism (ACM), to enhance the efficiency of OEM. These optimizations effectively address the challenges posed by data skewness, low cache efficiency, and high dimensionality. In addition, we establish a performance analysis model that characterizes the trade-off between memory consumption and performance improvement. We conducted extensive experiments to evaluate the performance of OEM. Compared to six state-of-the-art algorithms, namely REIN, Ada-REIN, TAMA, OpIndex, PEM and fgPEM, OEM achieves an average reduction in matching time by 40.4×, 38.5×, 12.0×, 33.1×, 27.8×, and 25.5×, respectively.