Better Behavioral Prediction, More Faithful Model Ablations? Evidence from Sequential Choice
Abstract
Using predictive models to explain cognition requires more than accurate behavioral predictions. Input ablations offer an appealing route: remove information from a model and interpret the resulting performance change as evidence of its importance for behavior. Yet this inference assumes that the model's dependence on information reflects the dependence of the process generating the behavior. We test it in two synthetic sequential bandit tasks with known generating policies, where past choices can remain informative when feedback is unavailable to a predictor. We compare GRUs and Transformers trained from scratch, a fine-tuned LLaMA model, and cognitive models across systematically varied reward contributions. Our analyses distinguish prediction after training without reward observations from the response of a fixed predictor to donor-reward replacement. Three findings emerge. First, in the restless task, neural models trained without rewards predict held-out choices better than four simple training-fitted behavioral baselines. Second, under matched donor replacement, accurate predictors can respond much less than the known generator. Third, at some reward weights, neural networks predict better than a pooled reinforcement-learning model but have less faithful changes in choice probabilities; the model ordering differs between the two tasks. These independent-test results separate information sufficient for prediction from response fidelity under a specified ablation in sequential choice. They motivate validating model-ablation responses independently of predictive performance before using them to infer how the observed behavior was generated.
Related
- Idiosyncratic Eye‐Movement Patterns to Faces Are Associated With Differences in Neural Processing: Evidence From Behavioral and Event‐Related Potential Responses
- Marmosets adaptively accumulate dynamic sequential evidence to make decisions
- scDisentangle: sequential disentanglement for single-cell perturbation response prediction
- Mere co-presence and evaluative observation differentially shape motor response execution and inhibition: Behavioral, signal-detection, and ERP evidence
- An orthographic advantage in spoken word recognition: Behavioral and ERP evidence from cross- and within-modal priming in Korean
BibTeX
@misc{xie2026better,
title = {{Better Behavioral Prediction, More Faithful Model Ablations? Evidence from Sequential Choice}},
author = {Hanbo Xie},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.36097},
eprint = {2609.36097},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.36097},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Better Behavioral Prediction, More Faithful Model Ablations? Evidence from Sequential Choice
AU - Hanbo Xie
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2609.36097
UR - https://arxiv.org/abs/2609.36097
AB - Using predictive models to explain cognition requires more than accurate behavioral predictions. Input ablations offer an appealing route: remove information from a model and interpret the resulting performance change as evidence of its importance for behavior. Yet this inference assumes that the model's dependence on information reflects the dependence of the process generating the behavior. We test it in two synthetic sequential bandit tasks with known generating policies, where past choices can remain informative when feedback is unavailable to a predictor. We compare GRUs and Transformers trained from scratch, a fine-tuned LLaMA model, and cognitive models across systematically varied reward contributions. Our analyses distinguish prediction after training without reward observations from the response of a fixed predictor to donor-reward replacement. Three findings emerge. First, in the restless task, neural models trained without rewards predict held-out choices better than four simple training-fitted behavioral baselines. Second, under matched donor replacement, accurate predictors can respond much less than the known generator. Third, at some reward weights, neural networks predict better than a pooled reinforcement-learning model but have less faithful changes in choice probabilities; the model ordering differs between the two tasks. These independent-test results separate information sufficient for prediction from response fidelity under a specified ablation in sequential choice. They motivate validating model-ablation responses independently of predictive performance before using them to infer how the observed behavior was generated.
ER -