Unstructured Log Analysis for System Anomaly Detection—A Study
semanticscholar
Abstract
Nowadays, with the rapid pace of innovation, a typical production infrastructure is getting huge, complicated and difficult to manage. Hence, incident detection and action have become a challenge to the operations and information security (InfoSec) teams. As we are moving toward deployments of complicated or complex large-scale micro-service architectures, the kind of data generated from all those systems is huge. So, it becomes very difficult to identify if anything goes wrong in underlying systems, i.e., the system is vulnerable to various attacks. Keeping track of the flow of traffic and user activities on a large scale, complicated environment is becoming very costly and unmanageable. Traditional systems and ways are becoming inefficient for zero-day security issues. So it is highly recommended to develop a system that is capable of raising an alarm for any detected anomaly after performing an automatic analysis of the generated logs. This study is conducted to review the research work on unstructured log analysis for the purpose of monitoring the system and anomaly detection. We have identified the datasets used for this purpose and also pointed out the challenges involved in unstructured data analysis.
MoreTranslated text
Key words
Log analysis, Micro-service, Log parsing, word2vec, temp2vec, Natural language processing
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined