How Optimality Structures Sparse Dictionaries: Theory for Interpreting SAE Representations
Abstract
Sparse Autoencoders (SAEs) have found success parsing neural network representations into interpretable concepts, providing a basis for understanding and control. However, what exactly SAEs extract and, hence, the scientific conclusions we can draw from them are not obvious. In short, if your SAE behaves strangely, does that reflect interesting neural network behaviour or an SAE-imposed distortion? Towards answering this, we use dictionary learning identifiability results to derive constraints that optimal dictionary learning features must satisfy. For example, an optimal feature will never turn on only while another is active. We use these conditions to explain various SAE oddities - hierarchical splitting & absorption, which features can be left in the residuals, dense antipodal features, and infinite feature splitting - simply as properties imposed by the dictionary learning objective. Finally, these constraints are diagnostic: real SAEs pass when measured on the dataset on which they were trained, but increasingly fail as the test dataset becomes more `distant'. In sum, we hope to provide theoretical tools to explain puzzling SAE patterns, allowing more principled inferences about internal model behaviour.
Related
- Sparse input representations explain odor discrimination in complex, concentration-varying mixtures
- Data-driven spectroscopic dictionaries and detector-calibrated inference for photon-limited Raman hyperspectral imaging of living cells
- OFC-SV: Interpreting structural variants in African and Asian cohorts with non-syndromic orofacial clefts
- The Core Promoter has an active role in interpreting patterning information
- Interpreting Seasonal Forecast Visualizations: Differences in Mental Models Between Experts and Users
BibTeX
@misc{dorrell2026optimality,
title = {{How Optimality Structures Sparse Dictionaries: Theory for Interpreting SAE Representations}},
author = {William Dorrell},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.02385},
eprint = {2606.02385},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.02385},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - How Optimality Structures Sparse Dictionaries: Theory for Interpreting SAE Representations
AU - William Dorrell
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2606.02385
UR - https://arxiv.org/abs/2606.02385
AB - Sparse Autoencoders (SAEs) have found success parsing neural network representations into interpretable concepts, providing a basis for understanding and control. However, what exactly SAEs extract and, hence, the scientific conclusions we can draw from them are not obvious. In short, if your SAE behaves strangely, does that reflect interesting neural network behaviour or an SAE-imposed distortion? Towards answering this, we use dictionary learning identifiability results to derive constraints that optimal dictionary learning features must satisfy. For example, an optimal feature will never turn on only while another is active. We use these conditions to explain various SAE oddities - hierarchical splitting & absorption, which features can be left in the residuals, dense antipodal features, and infinite feature splitting - simply as properties imposed by the dictionary learning objective. Finally, these constraints are diagnostic: real SAEs pass when measured on the dataset on which they were trained, but increasingly fail as the test dataset becomes more `distant'. In sum, we hope to provide theoretical tools to explain puzzling SAE patterns, allowing more principled inferences about internal model behaviour.
ER -