Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation
Abstract
Visual signals require compact yet sufficient representations for robust downstream prediction. Convolutional sparse coding (CSC) provides an explicit mechanism for suppressing redundant components while preserving signal content, but its sparsity coefficient is often manually selected during training. We propose a training-adaptive convolutional sparse coding framework for robust visual signal representation. Specifically, we unfold the CSC optimization with the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) and treat the sparsity coefficient as a differentiable variable jointly learned with the network parameters. From the information bottleneck perspective, this coefficient controls the trade-off between information retention and compression: the sparsity term promotes compact representations, while the reconstruction term together with task loss preserves task-relevant signal content. We further introduce a label-free post-training strategy that adjusts the compression strength for corrupted inputs with the main network parameters fixed. Experiments on CIFAR and ImageNet demonstrate competitive clean-data recognition and greatly improved robustness under different input perturbations.
Related
- What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control
- On the Information Bottleneck for Stable Variables
- Dynamics Aware Past-Future Information Bottleneck
- Optimal Multitask Representations Under a Rate Constraint: An Information Bottleneck Analysis
- On the Cardinality of Optimal Representations in the Binary-Source Information Bottleneck
BibTeX
@misc{qin2026trainingadaptive,
title = {{Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation}},
author = {Meng'en Qin and Yinchen Liu and Mingxuan Cui and Youlu Xing},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.19122},
eprint = {2609.19122},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.19122},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation
AU - Meng'en Qin
AU - Yinchen Liu
AU - Mingxuan Cui
AU - Youlu Xing
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2609.19122
UR - https://arxiv.org/abs/2609.19122
AB - Visual signals require compact yet sufficient representations for robust downstream prediction. Convolutional sparse coding (CSC) provides an explicit mechanism for suppressing redundant components while preserving signal content, but its sparsity coefficient is often manually selected during training. We propose a training-adaptive convolutional sparse coding framework for robust visual signal representation. Specifically, we unfold the CSC optimization with the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) and treat the sparsity coefficient as a differentiable variable jointly learned with the network parameters. From the information bottleneck perspective, this coefficient controls the trade-off between information retention and compression: the sparsity term promotes compact representations, while the reconstruction term together with task loss preserves task-relevant signal content. We further introduce a label-free post-training strategy that adjusts the compression strength for corrupted inputs with the main network parameters fixed. Experiments on CIFAR and ImageNet demonstrate competitive clean-data recognition and greatly improved robustness under different input perturbations.
ER -