Mechanistic Interpretability Needs a Logic of Inference
Abstract
Neural networks are increasingly studied not merely as predictive systems, but as complex systems whose internal computations are themselves objects of scientific inquiry. Recent progress in mechanistic interpretability (MI) has made it possible to identify neurons, features, circuits, population structures, and dynamical patterns associated with model behavior. As these tools improve, attention is shifting from what can be observed inside a model to how those observations support mechanistic explanations. We examine the parallel efforts of neuroscience and MI to address this question and identify recurring inferential bottlenecks when inferring mechanisms from complex neural systems. We propose five principles and organize them into a recursive logic for a more rigorous procedure of mechanistic inference. These principles provide a framework that supports scientific discovery in two senses: it helps turn local interpretability findings into cumulative knowledge of mechanisms within models, and it makes those mechanisms sufficiently precise to generate testable hypotheses about domains beyond the models. Evidence beyond MI is ultimately required to establish what model mechanisms reveal about the world.
Related
- Evo 2 as a classification machine: evidence from in-context learning and mechanistic interpretability
- Mechanistic Interpretability of Protein Language Models Reveals Encoded Structural and Functional Properties of Intrinsically Disordered Proteins
- Mechanistic gene-regulation-based trajectory inference with bowside
- GSET: Gene Signals from Embedding Trajectories select compact contexts for mechanistic GRN inference
- CardamomOT: A mechanistic optimal transport-based framework for gene regulatory network inference, trajectory reconstruction and generative modeling
BibTeX
@misc{hu2026mechanistic,
title = {{Mechanistic Interpretability Needs a Logic of Inference}},
author = {Zhimin Hu and Chen Jiang and Hanbo Xie and N Apurva Ratan Murty and Sashank Varma},
howpublished = {PsyArXiv},
year = {2026},
month = sep,
doi = {10.31234/osf.io/tjahb_v1},
url = {https://osf.io/preprints/psyarxiv/tjahb_v1/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Mechanistic Interpretability Needs a Logic of Inference
AU - Zhimin Hu
AU - Chen Jiang
AU - Hanbo Xie
AU - N Apurva Ratan Murty
AU - Sashank Varma
PY - 2026
DA - 2026/09/10
PB - PsyArXiv
DO - 10.31234/osf.io/tjahb_v1
UR - https://osf.io/preprints/psyarxiv/tjahb_v1/
AB - Neural networks are increasingly studied not merely as predictive systems, but as complex systems whose internal computations are themselves objects of scientific inquiry. Recent progress in mechanistic interpretability (MI) has made it possible to identify neurons, features, circuits, population structures, and dynamical patterns associated with model behavior. As these tools improve, attention is shifting from what can be observed inside a model to how those observations support mechanistic explanations. We examine the parallel efforts of neuroscience and MI to address this question and identify recurring inferential bottlenecks when inferring mechanisms from complex neural systems. We propose five principles and organize them into a recursive logic for a more rigorous procedure of mechanistic inference. These principles provide a framework that supports scientific discovery in two senses: it helps turn local interpretability findings into cumulative knowledge of mechanisms within models, and it makes those mechanisms sufficiently precise to generate testable hypotheses about domains beyond the models. Evidence beyond MI is ultimately required to establish what model mechanisms reveal about the world.
ER -