KDetect: Unsupervised Anomaly Detection for Cloud Systems Based on Time Series Clustering

To improve the user experience in Cloud systems, it is of major interest for Cloud management tools to be able to automatically detect and notify anomalies in the behavior of services executed in virtual machines in a non-intrusive manner. To this end, this paper presents KDetect, an unsupervised learning algorithm to detect anomalies in periodic time series using clustering techniques. Using unlabeled time series representing the resource consumption of an unknown service executing in a virtual machine as input, KDetect is able to identify patterns that correspond to anomalies in the behavior of that service, assuming that the service behaves normally most of the time. Our evaluation, run with data extracted from a production dataset and coming from virtual machines exhibiting very different normal and abnormal behaviors, shows that KDetect is able to achieve a very high accuracy in the detection of anomalies (with more than 98% of recall and precision in most cases).