A dynamic ensemble algorithm for anomaly detection in IoT imbalanced data streams

With the rapid development of ambient intelligence (AmI) in the Internet of Things (IoT), many data streams are generated from sensing devices in intelligent scenarios. Due to the deployment issues of IoT devices and the system's complexity, abnormal behavior is inevitable, resulting in imbalanced data categories. Moreover, data streams generated in IoT systems are dynamic, continuous, and as the environment changes, further increasing the difficulty of anomaly detection. Therefore, we model the monitored historical and current data from the perspective of dynamic imbalanced data streams classification to discover abnormal behaviors in IoT systems. In this paper we propose a dynamic ensemble algorithm for anomaly detection in IoT environments. First the abnormal data samples are synthesized by the borderline-synthetic minority over-sampling technique (Borderline-SMOTE) to relieve the sample imbalance problem. Then considering the dynamics and continuity of data streams we adopt a chunk-based strategy to train a LightGBM classifier for each chunk of data to adapt to the current data distribution. To improve the ensemble model's processing efficiency and anomaly detection accuracy we adopt a dynamic weighting strategy for base classifiers and remove the classifier whose accuracy performance is lower than the threshold. Finally we evaluate our proposed algorithm by conducting comparative experiments on real-world data streams. Experimental results show that our proposed algorithm outperforms the comparative anomaly detection methods in IoT scenarios.