Nethobench: Forecasting design and conditioning shape the structural realism of generated neural dynamics
- 1Netholabs, London, UK
Abstract
Forecasting models are increasingly used to model neural population activity, yet existing benchmarks typically evaluate them by pointwise predictive fidelity to held-out observations. In stochastic, partially observed systems, agreement with one sampled realization is distinct from preserving the statistical and dynamical structure of the process. We introduce Nethobench, a benchmark that evaluates structural agreement across five metric families: distributional, temporal--spectral, relational, geometric, and state-dynamical. We validate Nethobench using targeted perturbations and stochastic systems with known dynamics, and apply it to mouse calcium imaging, mouse Neuropixels recordings, and human intracranial recordings, using empirical references to calibrate interpretation. Nethobench detects targeted structural failures and assigns high scores to independent samples from the correct synthetic process despite trajectory mismatch. It tracks structural changes during training and under domain shift. Model comparisons reveal how forecasting formulation, architecture, training regime, and loss shape structural profiles, while OmniMouse input ablations improve all five family means when behavior is added to video in one held-out session. In Brain Treebank, analog retrieval passes more structural checks while a multistep GRU better predicts the observed realization. These results motivate reporting calibrated family-level structural metrics alongside pointwise accuracy, proper probabilistic scores, and task-specific endpoints when evaluating neural forecasts across species and recording modalities.
Related
- Striatal dopamine dynamics across temporal scales in brain-specific Atp2a2 heterozygous knockout mice during reward conditioning
- Bacterial Community Structural Dynamics and Predicted Functional Potential in Loxostege sticticalis
- Experimental and In Silico Analysis of the Structural Dynamics of Dengue NS2B-NS3 Protease
- From immature to mature epithelium: unveiling structural dynamics and transcriptional programs in rainbow trout intestinal barrier
- Resolution of chromosomal structural variation and vanA plasmid dynamics by nanopore-only sequencing across successive waves of a hospital outbreak of vancomycin-resistant Enterococcus faecium
BibTeX
@misc{marion2026nethobench,
title = {{Nethobench: Forecasting design and conditioning shape the structural realism of generated neural dynamics}},
author = {B. Marion and A. de Vicente and M. Mahn and A. Luthi and L. Luithlen and C. Mitelut},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.07.757249},
url = {https://www.biorxiv.org/content/10.64898/2026.10.07.757249v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Nethobench: Forecasting design and conditioning shape the structural realism of generated neural dynamics
AU - B. Marion
AU - A. de Vicente
AU - M. Mahn
AU - A. Luthi
AU - L. Luithlen
AU - C. Mitelut
PY - 2026
DA - 2026/10/07
PB - bioRxiv
DO - 10.64898/2026.10.07.757249
UR - https://www.biorxiv.org/content/10.64898/2026.10.07.757249v1
AB - Forecasting models are increasingly used to model neural population activity, yet existing benchmarks typically evaluate them by pointwise predictive fidelity to held-out observations. In stochastic, partially observed systems, agreement with one sampled realization is distinct from preserving the statistical and dynamical structure of the process. We introduce Nethobench, a benchmark that evaluates structural agreement across five metric families: distributional, temporal--spectral, relational, geometric, and state-dynamical. We validate Nethobench using targeted perturbations and stochastic systems with known dynamics, and apply it to mouse calcium imaging, mouse Neuropixels recordings, and human intracranial recordings, using empirical references to calibrate interpretation. Nethobench detects targeted structural failures and assigns high scores to independent samples from the correct synthetic process despite trajectory mismatch. It tracks structural changes during training and under domain shift. Model comparisons reveal how forecasting formulation, architecture, training regime, and loss shape structural profiles, while OmniMouse input ablations improve all five family means when behavior is added to video in one held-out session. In Brain Treebank, analog retrieval passes more structural checks while a multistep GRU better predicts the observed realization. These results motivate reporting calibrated family-level structural metrics alongside pointwise accuracy, proper probabilistic scores, and task-specific endpoints when evaluating neural forecasts across species and recording modalities.
ER -