Soon Hoe Lim, N. Benjamin Erichson, Francisco Utrera, Winnie Xu, Michael Mahoney
We introduce Noisy Feature Mixup (NFM), an inexpensive yet effective method for data augmentation that combines the best of interpolation based training and noise injection schemes. We use noise-perturbed convex combinations of datapoint pairs in both input and feature space to learn smoother decision boundaries, leading to improved robustness. The advantageous implicit regularization effects of NFM compared to previous mixup training methods are further understood with our theoretical insights. We show that residual networks and vision transfromers trained with NFM have favorable trade-offs between predictive accuracy and out-of-distribution generalization.
International Conference on Learning Representations (ICLR), 2022.