Research and application of streaming Data transmission and processing architecture based on Pulsar
2021 4th International Conference on Advanced Electronic Materials, Computers and Software Engineering (AEMCSE)(2021)
摘要
In order to solve the traditional message queue technology in the transmission of sensor data streams low transfer rate, high cost and lack of stream processing method, is proposed based on a stateless Pulsar message queue and the time sequence database IoTDB flow processing architecture, data transmission of hydrological data can provide efficient and reliable transmission, data preprocessing and early warning, provide the basis for subsequent data mining work. As a high performance message queue Kafka is often used in sensor data transmission scheme, Kafka is used to perform the same transmission task for comparative analysis. The experimental results show that Pulsar is able to perform the flow computation well, and has advantages in transmission speed and system resource utilization.
更多查看译文
关键词
Pulsar,message queue,streaming data,data preprocessing
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要