Towards desirable decision boundary by Moderate-Margin Adversarial Training

The previous adversarial training methods tended to use a larger uniform perturbation budget to obtain an inclusive decision boundary, which improved robustness. However, this large uniform perturbation budget will bring an unnecessary increase in the margin along adversarial directions, causing heavy cross-over between natural and adversarial examples. It is not conducive to balancing the trade-off between robustness and natural accuracy. In this paper, we propose a novel adversarial training scheme, namely Moderate-Margin Adversarial Training (MMAT), to achieve a better trade-off. Specifically, we generate finer-grained adversarial examples to mitigate the cross-over between them and natural examples of neighboring classes. Meanwhile, we design a hybrid loss to learn adversarial examples and natural examples respectively to further obtain a moderate decision boundary. Extensive experiments show MMAT achieves high natural accuracy and robustness under both black-box and white-box attacks. Especially, state-of-the-art robustness and natural accuracy are achieved on SVHN.