Characterizing Disk Health Degradation and Proactively Protecting Against Disk Failures for Reliable Storage Systems

The booming of cloud computing, online services and big data applications have resulted in dramatic expansion of storage systems. Meanwhile, disk drives are reported to be the most commonly replaced hardware component. Disk failures cause service downtime and even data loss, costing enterprises multi-trillion dollars per year. Existing disk failure management approaches are mostly reactive and incur high overheads. To overcome these problems, in this paper, we present a proactive, cost-effective solution to managing large-scale production storage systems. We aim to uncover the entire process in which disk's health deteriorates and forecast when disk drives will fail in the future. Due to a common lack of diagnostic information of disk failures, we rely on the Self-Monitoring, Analysis and Reporting Technology (SMART) data and explore statistical analysis techniques to identify the start of disk degradation. We then model the disk degradation processes as functions of SMART attributes, which eliminates the dependency on time and thus I/O workload. Experimental results from over 23,000 enterprise-class disk drives in a production data center show that our derived models can accurately quantify the degradation of disk health, which enables us to proactively protect data against disk failures. We also investigate several types of disk failures and propose remediation mechanisms to prolong disk lifetime.