LogMiner: A System Audit Log Reduction Strategy Based on Behavior Pattern Mining

Cyber attacks on enterprises and organizations have increased dramatically recently. Endpoint monitoring systems are widely deployed in enterprises and organizations to stop these attacks. System audit logs are the critical data resource of endpoint monitoring, containing trustworthy and comprehensive records for forensic investigation and behavior analysis. However, analyzing and understanding system audit logs is challenging due to the massive volume and low semantics of system events. This paper proposes LogMiner, a system audit log reduction approach based on behavior pattern mining. The key insight is that, user behavior exhibits unique and stable characteristics in system audit logs. Discovering the distinct pattern in system events and replacing them with behaviors provides a new log reduction strategy. LogMiner automatically extracts patterns of user behaviors from system events and identifies behaviors in audit logs upon these patterns. Experimental results on real-world data show that the reduction rate of LogMiner to system events is as high as 245.6x on average. In addition, LogMiner can accurately distinguish user behaviors and achieves an average F1 score of 100% on behavior identification.