ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on
Abstract
Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should therefore be audited for reliance on concepts such as acquisition artifacts and patient demographics. Current methods, including linear probes and concept activation vectors, measure concept reliance by asking whether each concept is decodable from a layer. We show that this framing is misleading: decodability reflects not only concept reliance but also correlations among concepts and the outcome in the auditing dataset. We reframe shortcut auditing as a decomposition of the variance of intermediary representations with Independent Canonical cONcepts (ICON). ICON quantifies the share of a layer's variance explained by each concept conditional on all other concepts and the outcome, and also reports the variance left unexplained. Because ICON explanations are comparable across layers, their trajectories distinguish concepts that drive predictions from those that are merely encoded in the representations. In simulated data with ground truth, ICON recovers concept importance more accurately than seven baselines. In skin-cancer and neuroimaging applications, it separates learned shortcuts from merely correlated concepts, which we confirm with independent validation checks.
Related
- Auditing bipartite motif interpretations: a worked example with conservation checks and open-path decomposition
- Low-Rank and Structured Sparse Tensor Decomposition for Anomaly Detection in Multivariate Functional Data
- Hearing the heat: a proof-of-concept study of nonlinear sound measures for distinguishing phases of rapid organic decomposition
- Compressing multivariate functions with tree tensor networks
- Inverse Cross-spectral Neural Networks for Multivariate Time Series
BibTeX
@misc{rane2026icon,
title = {{ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on}},
author = {Roshan Prakash Rane and Marco Simnacher and Manuel Pfeuffer and Marc-Andre Schulz and Nys Tjade Siegel and Maximilian Dreyer and Frederik Pahde and Wojciech Samek and Sonja Greven and Kerstin Ritter},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2608.26083},
eprint = {2608.26083},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.26083},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on
AU - Roshan Prakash Rane
AU - Marco Simnacher
AU - Manuel Pfeuffer
AU - Marc-Andre Schulz
AU - Nys Tjade Siegel
AU - Maximilian Dreyer
AU - Frederik Pahde
AU - Wojciech Samek
AU - Sonja Greven
AU - Kerstin Ritter
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2608.26083
UR - https://arxiv.org/abs/2608.26083
AB - Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should therefore be audited for reliance on concepts such as acquisition artifacts and patient demographics. Current methods, including linear probes and concept activation vectors, measure concept reliance by asking whether each concept is decodable from a layer. We show that this framing is misleading: decodability reflects not only concept reliance but also correlations among concepts and the outcome in the auditing dataset. We reframe shortcut auditing as a decomposition of the variance of intermediary representations with Independent Canonical cONcepts (ICON). ICON quantifies the share of a layer's variance explained by each concept conditional on all other concepts and the outcome, and also reports the variance left unexplained. Because ICON explanations are comparable across layers, their trajectories distinguish concepts that drive predictions from those that are merely encoded in the representations. In simulated data with ground truth, ICON recovers concept importance more accurately than seven baselines. In skin-cancer and neuroimaging applications, it separates learned shortcuts from merely correlated concepts, which we confirm with independent validation checks.
ER -