Log Analytics In Hpc: A Data-Driven Reinforcement Learning Framework

High Performance Computing (HPC) has been employed in many fields such as aerospace, weather forecast, numerical simulation, scientific research etc. Security of HPC, especially anomaly/intrusion detection, has attracted many attentions in recent years. Given the heavily instrumented property of HPC systems, logs become an effective and direct data source that can be utilized to evaluate the system status, further, to detect anomalies or malicious users. In this paper, we offer a novel perspective, treating the anomaly detection in HPC as a sequential decision process, and further applying reinforcement learning techniques to learn the state transition process, based on which we build a framework named as ReLog to detect anomalies or malicious users. Besides, a common challenge of employing machine learning techniques is lacking sufficient data, we provide a generative adversarial network (GAN)-based solution to generate sufficient training data in HPC. The experimental validations are conducted based on real-world collected MPI logs, and our results demonstrate a 93% of detection accuracy on the collected dataset.