A Unified Search Framework for Data Augmentation and Neural Architecture on Small-Scale Image Data Sets

Data augmentation is an effective technique to enrich the training data's diversity and reduce the risk of overfitting. However, different data sets have distinct preferences on various augmentation techniques. Recently, automated data augmentation (auto-augmentation), which could engineer augmentation policy automatically, drew a growing interest. Previous auto-augmentation methods usually utilize a density matching (DM) strategy, which highly depends on a large data scale to ensure a precise policy evaluation. When facing small-scale data sets, it usually achieves an inferior performance. To address the problem, an improved method named Augmented DM is proposed by augmenting the train data with policies uniformly sampled from a prior distribution, making the policy evaluation more precise. Moreover, we propose a unified search framework for data augmentation and neural architecture (USAA) by formulating the search processes with one formulation. As a result, both optimal augmentation policy and neural architecture could be obtained within one round of the search process. Extensive experiments have been conducted on a bag of medical data sets with general small scales, and the results show that the proposed Augmented DM and USAA can outperform the state-of-the-art auto-augmentation and AutoML approaches, respectively.