Learning Interpretable Switching Dynamics in Shared Neural-Behavioral Latent Space
- 1Yale University
Abstract
Modern recording technologies enable simultaneous measurement of high-dimensional neural activity and rich behavioral variables, creating both an opportunity and a modeling challenge: identifying the shared latent structure that mediates the brain-behavior relationship and the underlying dynamics through which it evolves across behavioral states. Existing approaches focus on different parts of this challenge: shared representation learning methods identify neural-behavioral subspaces but typically do not model dynamical structure, while dynamical models capture temporal variation and discrete state transitions in neural activity, but do not explicitly disentangle shared neural-behavioral structure from modality-specific variability. We introduce Switching Shared Latent Dynamics (SSLD), a unified framework that learns a latent representation shared between neural activity and behavior and endows this shared space with nonlinear switching recurrent dynamics to reflect the hypothesis that behaviorally relevant neural representations exhibit structured dynamical changes across behavioral states. Complementary private latent variables capture modality-specific variability, providing a window into neural activity that does not directly interact with behavior. We evaluate SSLD on a simulated dataset and four diverse experimental datasets spanning species, brain regions, and recording modalities: monkey motor and premotor cortex during reaching, somatosensory cortex during a bump task, widefield calcium imaging across mouse dorsal cortex during (a) self-initiated decision-making and (b) spontaneous movements. Across all four datasets, SSLD accurately reconstructs neural and behavioral signals, recovers discrete states in the dynamics that align with experimentally defined behavioral epochs, and isolates behaviorally-relevant neural information in the shared latent while preserving private neural variability. Ablations confirm that shared representation learning, behavioral supervision, and switching dynamics each contribute to performance. SSLD offers an interpretable approach to modeling shared neural-behavioral dynamics that complements ongoing efforts toward foundation models for neuroscience.
Related
- Host behavioral responses to perceived risk shape spatial disease dynamics
- Integrative Advances in Honey Bee Gut–Brain Interactions: From Microbiota Dynamics to Behavioral Regulation and Colony Health
- Intrinsic network dynamics catalyze switching in functional connectivity states of the brain
- Repeated within-session rule switching in the marmoset: a paradigm for tracking the dynamics of cognitive flexibility
- Inferring Multi-Timescale Neural Dynamics with Switching Linear Dynamical Systems
BibTeX
@misc{zhang2026learning,
title = {{Learning Interpretable Switching Dynamics in Shared Neural-Behavioral Latent Space}},
author = {Y. Zhang and J. O. Caro and R. Oren and M. J. Higley and J. Cardin and A. Churchland and S. Saxena},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.02.756323},
url = {https://www.biorxiv.org/content/10.64898/2026.10.02.756323v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Learning Interpretable Switching Dynamics in Shared Neural-Behavioral Latent Space
AU - Y. Zhang
AU - J. O. Caro
AU - R. Oren
AU - M. J. Higley
AU - J. Cardin
AU - A. Churchland
AU - S. Saxena
PY - 2026
DA - 2026/10/02
PB - bioRxiv
DO - 10.64898/2026.10.02.756323
UR - https://www.biorxiv.org/content/10.64898/2026.10.02.756323v1
AB - Modern recording technologies enable simultaneous measurement of high-dimensional neural activity and rich behavioral variables, creating both an opportunity and a modeling challenge: identifying the shared latent structure that mediates the brain-behavior relationship and the underlying dynamics through which it evolves across behavioral states. Existing approaches focus on different parts of this challenge: shared representation learning methods identify neural-behavioral subspaces but typically do not model dynamical structure, while dynamical models capture temporal variation and discrete state transitions in neural activity, but do not explicitly disentangle shared neural-behavioral structure from modality-specific variability. We introduce Switching Shared Latent Dynamics (SSLD), a unified framework that learns a latent representation shared between neural activity and behavior and endows this shared space with nonlinear switching recurrent dynamics to reflect the hypothesis that behaviorally relevant neural representations exhibit structured dynamical changes across behavioral states. Complementary private latent variables capture modality-specific variability, providing a window into neural activity that does not directly interact with behavior. We evaluate SSLD on a simulated dataset and four diverse experimental datasets spanning species, brain regions, and recording modalities: monkey motor and premotor cortex during reaching, somatosensory cortex during a bump task, widefield calcium imaging across mouse dorsal cortex during (a) self-initiated decision-making and (b) spontaneous movements. Across all four datasets, SSLD accurately reconstructs neural and behavioral signals, recovers discrete states in the dynamics that align with experimentally defined behavioral epochs, and isolates behaviorally-relevant neural information in the shared latent while preserving private neural variability. Ablations confirm that shared representation learning, behavioral supervision, and switching dynamics each contribute to performance. SSLD offers an interpretable approach to modeling shared neural-behavioral dynamics that complements ongoing efforts toward foundation models for neuroscience.
ER -