Data Removal from an AUC Optimization Model

Our learned model may be required to make some dynamic adjustments owing to data removals in privacy, adversarial learning, etc. Previous studies on this issue mostly focus on the standard classification accuracy. This work takes one step on data removal for AUC optimization, where previous methods can not be applied directly since AUC is measured by a sum of losses defined over pairs of instances from different classes. We develop the Data Removal algorithm for AUC optimization (DRAUC), and the basic idea is to adjust the trained model according to the removed data, rather than retrain another model again from the scratch. Our algorithm only needs to maintain some data statistics, without storing the training data in memory. For high-dimensional data, we utilize the frequent direction algorithm to approximate the second-order statistics, and solve the numerical solution based on gradient descent so as to avoid calculating the inverse of Hessian matrix. We verify the effectiveness of the proposed DRAUC both theoretically and empirically.