A Framework for Analyzing Skew in Evaluation Metrics

For several evaluation metrics for classification problems, correctly classifying an additional point from one class will have a different effect on the value of the evaluation metric compared to correctly classifying an additional point from another class. In this paper, we describe a method for quanti- fying these effects based on " metric skew". After describing how to find the skew for each class given a particular evalua- tion metric, we show what the skews are for several common evaluation metrics. In particular, we show that these skews provide a new viewpoint on metrics from which previously known as well as new properties about several popular met- rics can be observed.