What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions
Abstract
Sparse autoencoder (SAE) features are typically discovered from how their activations co-vary with behaviour, yet they are often used through interventions such as steering or ablation. We take intervention attribution, the first-order effect of a specified intervention on the model output, as our estimand. For a broad class of response functions, we show that observational feature-discovery evidence generally does not identify this attribution, and derive an exact gradient-based decomposition of the gap between linear-probe predictions and intervention attribution. Across four model-SAE pairs, median probe-to-attribution magnitude ratios range from tens to hundreds, while attributions closely track small-step steering. For single-latent ablations with nonzero effects, gradient predictions match the realized sign in 91-97% of cases, compared with 17-35% for probe-implied changes. Attribution-selected sets also produce larger absolute mean ablation effects on held-out prompts than sets selected by correlation, rank tests or probe coefficients. Together, these results establish intervention attribution as a principled, gradient-estimable target for selecting SAE features for intervention.
Related
- Model-free and Distributionally Robust Feature Screening with False Discovery Control for High-Dimensional Heterogeneous Data
- Interpretable Feature Discovery from Unstructured Data Using LLMs and Statistical Choice Models
- Proxy-Adjusted Causal Discovery from Targeted Interventions
- Statistical Inference for Causal Discovery under Selection and Latent Variables via Single-Target Interventions
- Radon Measure Representations for Infinite-Width Neural Networks with Singular Activations
BibTeX
@misc{guo2026what,
title = {{What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions}},
author = {Yifeng Guo and Yanran Li and Xiang Chen},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.15850},
eprint = {2610.15850},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.15850},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions
AU - Yifeng Guo
AU - Yanran Li
AU - Xiang Chen
PY - 2026
DA - 2026/10/11
PB - arXiv
DO - 10.48550/arxiv.2610.15850
UR - https://arxiv.org/abs/2610.15850
AB - Sparse autoencoder (SAE) features are typically discovered from how their activations co-vary with behaviour, yet they are often used through interventions such as steering or ablation. We take intervention attribution, the first-order effect of a specified intervention on the model output, as our estimand. For a broad class of response functions, we show that observational feature-discovery evidence generally does not identify this attribution, and derive an exact gradient-based decomposition of the gap between linear-probe predictions and intervention attribution. Across four model-SAE pairs, median probe-to-attribution magnitude ratios range from tens to hundreds, while attributions closely track small-step steering. For single-latent ablations with nonzero effects, gradient predictions match the realized sign in 91-97% of cases, compared with 17-35% for probe-implied changes. Attribution-selected sets also produce larger absolute mean ablation effects on held-out prompts than sets selected by correlation, rank tests or probe coefficients. Together, these results establish intervention attribution as a principled, gradient-estimable target for selecting SAE features for intervention.
ER -