Designing a Methodological Framework for the Empirical Evaluation of Self-Protecting Systems

Increasingly, cyber attacks against enterprises and governments make use of automated tools. For this reason, and given the importance of a timely protection, in the last decade there has been a push in researching methodologies to automate the full defense life-cycle of computer systems. The two core phases of this life-cycle are Intrusion Detection and Intrusion Response. However, while some progress has been done on the former, the latter is still at an early stage. This is due to several factors, among which the lack of a standardized methodology for the validation and comparison of Intrusion Response methodologies. In this paper, we attempt to fill this gap by introducing a methodological framework for the quantitative empirical evaluation of self-protecting systems, based on the metrics of response time and cost. An experimental design is also provided and its applicability is illustrated by the means of a template experiment.