Training More Robust Classification Model via Discriminative Loss and Gaussian Noise Injection
Abstract
Robustness of deep neural networks to input noise remains a critical challenge, as naive noise injection often degrades accuracy on clean (uncorrupted) data. We propose a novel training framework that addresses this trade-off through two complementary objectives. First, we introduce a loss function applied at the penultimate layer that explicitly enforces intra-class compactness and increases the margin to analytically defined decision boundaries. This enhances feature discriminativeness and class separability for clean data. Second, we propose a class-wise feature alignment mechanism that brings noisy data clusters closer to their clean counterparts. Furthermore, we provide a theoretical analysis demonstrating that improving feature stability under additive Gaussian noise implicitly reduces the curvature of the softmax loss landscape in input space, as measured by Hessian eigenvalues.This thus naturally enhances robustness without explicit curvature penalties. Conversely, we also theoretically show that lower curvatures lead to more robust models. We validate the effectiveness of our method on standard benchmarks and our custom dataset. Our approach significantly reinforces model robustness to various perturbations while maintaining high accuracy on clean data, advancing the understanding and practice of noise-robust deep learning.
Related
- Adversarially Trained Linear Transformers Are Optimal Robust In-Context Learners for Gaussian Mixtures
- Conservative neural posterior estimation via distributionally robust training
- Minimax Rates for Robust Gaussian Mean Estimation Near the Breakdown Point under Constraints
- Robust topology optimization with non-Gaussian material fields using polygonal finite elements
- Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation
BibTeX
@misc{nguyen2026training,
title = {{Training More Robust Classification Model via Discriminative Loss and Gaussian Noise Injection}},
author = {Hai-Vy Nguyen and Fabrice Gamboa and Sixin Zhang and Reda Chhaibi and Serge Gratton and Thierry Giaccone},
howpublished = {Transactions on Machine Learning Research},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2405.18499},
eprint = {2405.18499},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05562099v1},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Training More Robust Classification Model via Discriminative Loss and Gaussian Noise Injection
AU - Hai-Vy Nguyen
AU - Fabrice Gamboa
AU - Sixin Zhang
AU - Reda Chhaibi
AU - Serge Gratton
AU - Thierry Giaccone
PY - 2026
DA - 2026/10/19
PB - Transactions on Machine Learning Research
DO - 10.48550/arxiv.2405.18499
UR - https://hal.science/hal-05562099v1
AB - Robustness of deep neural networks to input noise remains a critical challenge, as naive noise injection often degrades accuracy on clean (uncorrupted) data. We propose a novel training framework that addresses this trade-off through two complementary objectives. First, we introduce a loss function applied at the penultimate layer that explicitly enforces intra-class compactness and increases the margin to analytically defined decision boundaries. This enhances feature discriminativeness and class separability for clean data. Second, we propose a class-wise feature alignment mechanism that brings noisy data clusters closer to their clean counterparts. Furthermore, we provide a theoretical analysis demonstrating that improving feature stability under additive Gaussian noise implicitly reduces the curvature of the softmax loss landscape in input space, as measured by Hessian eigenvalues.This thus naturally enhances robustness without explicit curvature penalties. Conversely, we also theoretically show that lower curvatures lead to more robust models. We validate the effectiveness of our method on standard benchmarks and our custom dataset. Our approach significantly reinforces model robustness to various perturbations while maintaining high accuracy on clean data, advancing the understanding and practice of noise-robust deep learning.
SN - 2835-8856
ER -