Cortico-subcortical multi-head self-attention as a substrate for cognitive performance
- 1University of Bern
Abstract
The neocortex is central to mammalian cognition, yet a computational framework that is both biologically constrained and capable of performing complex cognitive tasks remains missing. Here we show that cortico-thalamic circuits are well suited to implement multi-head self- and cross-attention, the mechanism underlying the cognitive abilities of transformer networks. We propose that layer 2/3 pyramidal cells maintain a recurrent key-value memory, while layer 5 pyramidal cells decode the memory retrieved by an incoming query. The computation of keys, values and queries maps onto core and matrix thalamo-cortical projections, distributed across the micro- and macro-columns of a cortical area. One cortical area forms an attention head, and cortex a multi-head self-attention network. The same thalamo-cortical microcircuit also calculates sensory prediction errors guiding gradient-based synaptic plasticity. A reward-prediction error gates via basal ganglia the cortical output and the re-activation of hippocampal memories. The trained network aligns with human intracranial recordings during speech perception. Overall, the suggested cortico-subcortical attention circuit may represent a substrate for the cognitive capacity of mammals.
Related
- MICROSTRUCTURAL ORGANIZATION OF CORTICO-SUBCORTICAL FUNCTIONAL CONVERGENCE
- Dopamine Depletion Drives Whole-Brain Oscillatory Disruptions via Cortico-Subcortical Resonance: A Multiscale Model of Parkinson's Disease in Mice
- Diminished ascending drive in transdiagnostic anhedonia shifts the cortico-subcortical hierarchy away from bottom-up dominance
- Transferable conditional brain charts link cortico-subcortical structure to Alzheimer's disease progression
- Strong mnemonic prediction errors increase cognitive control, attention, and arousal.
BibTeX
@misc{babu2026corticosubcortical,
title = {{Cortico-subcortical multi-head self-attention as a substrate for cognitive performance}},
author = {V. K. Babu and A. Granier and T. Proix and M. Woodman and V. Jirsa and T. Balvocius and A. Saudargiene and W. Senn},
howpublished = {bioRxiv},
year = {2026},
month = aug,
doi = {10.64898/2026.08.06.743205},
url = {https://www.biorxiv.org/content/10.64898/2026.08.06.743205v2},
note = {CC BY},
}
RIS
TY - UNPB
TI - Cortico-subcortical multi-head self-attention as a substrate for cognitive performance
AU - V. K. Babu
AU - A. Granier
AU - T. Proix
AU - M. Woodman
AU - V. Jirsa
AU - T. Balvocius
AU - A. Saudargiene
AU - W. Senn
PY - 2026
DA - 2026/08/06
PB - bioRxiv
DO - 10.64898/2026.08.06.743205
UR - https://www.biorxiv.org/content/10.64898/2026.08.06.743205v2
AB - The neocortex is central to mammalian cognition, yet a computational framework that is both biologically constrained and capable of performing complex cognitive tasks remains missing. Here we show that cortico-thalamic circuits are well suited to implement multi-head self- and cross-attention, the mechanism underlying the cognitive abilities of transformer networks. We propose that layer 2/3 pyramidal cells maintain a recurrent key-value memory, while layer 5 pyramidal cells decode the memory retrieved by an incoming query. The computation of keys, values and queries maps onto core and matrix thalamo-cortical projections, distributed across the micro- and macro-columns of a cortical area. One cortical area forms an attention head, and cortex a multi-head self-attention network. The same thalamo-cortical microcircuit also calculates sensory prediction errors guiding gradient-based synaptic plasticity. A reward-prediction error gates via basal ganglia the cortical output and the re-activation of hippocampal memories. The trained network aligns with human intracranial recordings during speech perception. Overall, the suggested cortico-subcortical attention circuit may represent a substrate for the cognitive capacity of mammals.
ER -