A probabilistic-based framework for infosec alert correlation

Deploying a large number of information security (INFOSEC) systems can provide in-depth protection for systems and networks. However, the sheer number of security alerts output by security sensors can overwhelm security analysts from performing effective analysis and taking timely response. Therefore, alert correlation is the core component in a security management system. Most of existing alert correlation techniques depend on a priori and hard-coded domain knowledge that lead to their limited capabilities of detecting new attack strategies. These approaches also focus more on the aggregation and analysis of raw security alerts, and build basic or low-level attack scenarios. This thesis focuses on discovering novel attack strategies with analysis of security alerts. Our framework helps security administrator aggregate redundant alerts, intelligently correlate security alerts, analyze attack strategies, and take appropriate actions against forthcoming attacks. In alert correlation, we have developed an integrated correlation system with three complementary correlation mechanisms. We have developed a probabilistic-based correlation engine that incorporates domain knowledge to correlate alerts that have direct causal relationship. We have developed a statistical analysis-based and a temporal analysis-based correlation engine to discover attack transition patterns in which attack steps do not have direct causal relationship in terms of security and performance measure but exhibit statistical and temporal patterns. We construct attack scenarios and conduct attack path analysis based on the correlation results. Security analysts are presented with aggregated information on attack strategies from the integrated correlation system. In attack plan recognition, we address the challenges of identifying attacker's high-level strategies and intentions as well as predicting upcoming attacks. We apply graph-based techniques to correlating isolated attack scenarios derived from low-level alert correlation based on their relationship in attack plans. We conduct probabilistic inference to evaluate the likelihood of attack goal(s) and predict potential upcoming attacks based on observed attack activities. We evaluate our algorithms using DARPA's Grand Challenge Problem (GCP) data sets and live traffic data collected from our backbone network. The results show that our approach can effectively discover novel attack strategies, provide a quantitative analysis of attack scenarios and identify attack plans.