When Model Complexity Matters in Cross-Subject Naturalistic fMRI Prediction

Abstract

Increasingly complex neural models predict blood-oxygen-level-dependent (BOLD) responses to naturalistic stimuli, but the predictive value of this added complexity remains unclear. We ask when pretrained stimulus representations help and how much downstream complexity is needed, with and without recent measured BOLD history. We compare linear diagnostic references and nonlinear predictors across HCP movie fMRI (154 subjects), BOLD Moments (10 subjects), and Algonauts 2025 (4 subjects). Under a shared offline protocol, models are fitted on training subjects and evaluated on held-out subjects viewing largely familiar stimuli, without refitting. Pretrained visual features generally improve prediction over raw pixels; for linear autoregression, this advantage becomes more pronounced over longer rollouts (recursive predictions without additional measured BOLD). Measured BOLD history provides substantial gains that generally diminish {as rollouts lengthen}. Nonlinear stimulus-only predictors can improve on linear readouts, whereas linear autoregression provides a strong history-conditioned baseline. Controls examine context length, regularization, fitting procedures, and model capacity. Regional response consistency across subjects is associated with held-out prediction accuracy, while less consistent regions tend to gain more from measured history. In paired simulations with fixed linear neural dynamics, slower hemodynamic responses generally sustain high temporal prediction correlations over longer rollouts, illustrating how the observation process can contribute to predictive persistence. Together, these findings support evaluating stimulus representations and downstream complexity separately, using history-conditioned linear prediction as a diagnostic reference under matched inputs and measurement availability.

Latent Prediction Yields Cross-Subject Alignable Representations in Sensory-Evoked fMRI
Serwan Jassim, Victoria Bosch, Daniel Anthes +2
arXiv29 Oct 2026PreprintCC BY 4.0Full textNeurosciencearXiv
Sex differences in pediatric EEG inter-subject correlation during naturalistic movie watching: a large-scale characterization of the Healthy Brain Network EEG dataset
E. Tomar, A. Silvan, E. Y. Fu +2
bioRxiv7 Sep 2026PreprintCC BY-NC-NDFull textNeurosciencebioRxiv
Longitudinal Subject Pairing in Cross-Sectional Neonatal Data Reveals Asynchronous Structural and Functional Brain Maturation
R. Namiranian, M. Sadeghi, H. Abrishami Moghaddam
bioRxiv8 Jul 2026PreprintAll rights reservedFull textNeurosciencebioRxiv
Do sEMG Foundation Models Improve Cross-Subject Generalisation?
William Raftery, Dario Farina
arXiv5 Nov 2026PreprintCC BY 4.0Full textNeurosciencearXiv
A longitudinal naturalistic fMRI study of trait anxiety and cross-phase consistency in subjective fear signature expression
C.-L. W. Chen, F.-C. B. Chou, P.-Y. A. Hsiao +1
bioRxiv24 Jul 2026PreprintCC BY-NCFull textNeurosciencebioRxiv

Cite

BibTeX RIS

BibTeX

@misc{zhao2026when,
  title = {{When Model Complexity Matters in Cross-Subject Naturalistic fMRI Prediction}},
  author = {Junhao Zhao and Jacob Kang and Colin Kurniawan and David Simberg and Nan Xu},
  howpublished = {arXiv},
  year = {2026},
  month = nov,
  doi = {10.48550/arxiv.2611.04041},
  eprint = {2611.04041},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04041},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - When Model Complexity Matters in Cross-Subject Naturalistic fMRI Prediction
AU  - Junhao Zhao
AU  - Jacob Kang
AU  - Colin Kurniawan
AU  - David Simberg
AU  - Nan Xu
PY  - 2026
DA  - 2026/11/02
PB  - arXiv
DO  - 10.48550/arxiv.2611.04041
UR  - https://arxiv.org/abs/2611.04041
AB  - Increasingly complex neural models predict blood-oxygen-level-dependent (BOLD) responses to naturalistic stimuli, but the predictive value of this added complexity remains unclear. We ask when pretrained stimulus representations help and how much downstream complexity is needed, with and without recent measured BOLD history. We compare linear diagnostic references and nonlinear predictors across HCP movie fMRI (154 subjects), BOLD Moments (10 subjects), and Algonauts 2025 (4 subjects). Under a shared offline protocol, models are fitted on training subjects and evaluated on held-out subjects viewing largely familiar stimuli, without refitting. Pretrained visual features generally improve prediction over raw pixels; for linear autoregression, this advantage becomes more pronounced over longer rollouts (recursive predictions without additional measured BOLD). Measured BOLD history provides substantial gains that generally diminish {as rollouts lengthen}. Nonlinear stimulus-only predictors can improve on linear readouts, whereas linear autoregression provides a strong history-conditioned baseline. Controls examine context length, regularization, fitting procedures, and model capacity. Regional response consistency across subjects is associated with held-out prediction accuracy, while less consistent regions tend to gain more from measured history. In paired simulations with fixed linear neural dynamics, slower hemodynamic responses generally sustain high temporal prediction correlations over longer rollouts, illustrating how the observation process can contribute to predictive persistence. Together, these findings support evaluating stimulus representations and downstream complexity separately, using history-conditioned linear prediction as a diagnostic reference under matched inputs and measurement availability.
ER  -

Abstract

Related

Cite