FlyHack: Stress-Testing Mechanistic Claims in a Connectome-Constrained Visual Model
Abstract
Connectome-constrained neural networks make biological components directly accessible to intervention, but their mechanistic conclusions can still depend on the comparison model, intervention, circuit representation and trained solution. We stress-test these dependencies in flyvis, a \emph{Drosophila} motion-vision model, using its 50-model ensemble, task-matched causal CNN controls and replicated interventions. At the level of model behaviour, three-level reverse-phi reversal in flyvis is strongly period- and velocity-dependent and also appears in task-matched CNNs; neither family reproduces the fly's stationary-sawtooth response. Adversarial direction reversal requires a perturbation bound of 0.21--0.27 times stimulus contrast in flyvis versus 0.13--0.25 in CNNs, but the gap over a parameter-matched CNN largely closes for larger CNNs. At the level of mechanism, matching attribution to the intervention matters: for held-out-mean replacement, first-order edge attribution patching reaches pooled AUROC 0.970 and Spearman 0.872, versus 0.933 and 0.126 when its zero-aligned score is evaluated against the same ground truth. Circuit conclusions also vary with representation and ensemble member. Greedy cell-type sets meet a model-relative T4/T5 tuning criterion in 25 of 50 models, while T4/T5 output silencing reduces the tested optic-flow behaviour to at most two of eight clips in 40. In six models where the cell-type search first succeeds, the corresponding connection sets can be pruned to 13--31 added classes; under an additional two-sided response-amplitude check, only two of those six first-success cell-type sets still meet the tuning threshold. These results support a narrower use of connectome-constrained models: they make mechanistic hypotheses directly testable, but the resulting claims must be scoped to the model family, intervention, representation and trained solution.
Related
- Discovering Subtypes of Neurodegenerative Progression with a Scalable Connectome-Constrained Dynamic Model
- ApproxBrain: Certified Sufficiency Frontiers in Connectome-Constrained Neural Systems
- Sex-dependent effects of chronic stress during adolescence on cognitive bias and functional connectome in young adult rats
- The (Butter)fly Dream: Linear Separability of World-Model Rollouts in a Connectome-Constrained Visual Model
- Representational geometry as a fidelity metric for connectome-constrained networks: evidence from the Drosophila visual system
BibTeX
@misc{dhawan2026flyhack,
title = {{FlyHack: Stress-Testing Mechanistic Claims in a Connectome-Constrained Visual Model}},
author = {Aashish Dhawan},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.13018},
eprint = {2610.13018},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.13018},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - FlyHack: Stress-Testing Mechanistic Claims in a Connectome-Constrained Visual Model
AU - Aashish Dhawan
PY - 2026
DA - 2026/10/09
PB - arXiv
DO - 10.48550/arxiv.2610.13018
UR - https://arxiv.org/abs/2610.13018
AB - Connectome-constrained neural networks make biological components directly accessible to intervention, but their mechanistic conclusions can still depend on the comparison model, intervention, circuit representation and trained solution. We stress-test these dependencies in flyvis, a \emph{Drosophila} motion-vision model, using its 50-model ensemble, task-matched causal CNN controls and replicated interventions. At the level of model behaviour, three-level reverse-phi reversal in flyvis is strongly period- and velocity-dependent and also appears in task-matched CNNs; neither family reproduces the fly's stationary-sawtooth response. Adversarial direction reversal requires a perturbation bound of 0.21--0.27 times stimulus contrast in flyvis versus 0.13--0.25 in CNNs, but the gap over a parameter-matched CNN largely closes for larger CNNs. At the level of mechanism, matching attribution to the intervention matters: for held-out-mean replacement, first-order edge attribution patching reaches pooled AUROC 0.970 and Spearman 0.872, versus 0.933 and 0.126 when its zero-aligned score is evaluated against the same ground truth. Circuit conclusions also vary with representation and ensemble member. Greedy cell-type sets meet a model-relative T4/T5 tuning criterion in 25 of 50 models, while T4/T5 output silencing reduces the tested optic-flow behaviour to at most two of eight clips in 40. In six models where the cell-type search first succeeds, the corresponding connection sets can be pruned to 13--31 added classes; under an additional two-sided response-amplitude check, only two of those six first-success cell-type sets still meet the tuning threshold. These results support a narrower use of connectome-constrained models: they make mechanistic hypotheses directly testable, but the resulting claims must be scoped to the model family, intervention, representation and trained solution.
ER -