Policy regularization as a unifying theory of the striatal division of labor in learning
- 1Harvard University
Abstract
Learning novel behaviors requires balancing previously learned actions with the ability to flexibly adapt to changing reward contingencies. This trade-off is well documented in the division of labor between dorsolateral striatum (DLS), which promotes selection of cached, history-dependent actions, and dorsomedial striatum (DMS), which supports flexible learning as reward contingencies change. Here, we propose policy regularization as a general computational principle for understanding this division. Capacity-limited agents face a fundamental trade-off between maximizing reward and minimizing the cost of deviating from a default policy that caches frequently used action transitions. We formalize this trade-off as a KL-regularized reward objective in which a flexible controller (DMS) incurs a cost for diverging from a history-dependent default (DLS). The resulting optimal policy is a combination of a reward-driven action value, continuously updated by DMS, and a default policy conditioned on action history, cached by DLS. In practice, DLS consolidates the action transitions shaped by DMS's reward-driven value learning, so DLS preserves behaviors that were once optimal even when reward contingencies change, producing robust but inflexible action selection and effectively "regularizing" DMS-driven learning. We show that a single model with one shared parameter set and consistent lesion rules provides a unifying explanation for the functional organization of the striatum, reproducing canonical DLS-DMS dissociations in outcome devaluation, serial spatial reversal, skilled action sequencing, and motor sequence execution tasks.
Related
- Punishment Enhances Reward Learning through Striatal Prediction Errors
- Distinct amygdala-striatal and amygdala-cortical circuits encode and retrieve predictive learning to guide choice and decision-making
- Dopamine release-associated striatal blood oxygenation and whole-brain dynamics during associative learning
- Striatal and orbitofrontal grey matter volumes are associated with cognitive event related potentials during associative learning task performance in male and female children and adolescents
- Semaglutide Increases Striatal Leptin Sensitivity, Impacting Dopaminergic Response to Cocaine
BibTeX
@misc{bhatia2026policy,
title = {{Policy regularization as a unifying theory of the striatal division of labor in learning}},
author = {C. Bhatia and S. J. Gershman and L. Lai},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.10.750654},
url = {https://www.biorxiv.org/content/10.64898/2026.09.10.750654v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - Policy regularization as a unifying theory of the striatal division of labor in learning
AU - C. Bhatia
AU - S. J. Gershman
AU - L. Lai
PY - 2026
DA - 2026/09/10
PB - bioRxiv
DO - 10.64898/2026.09.10.750654
UR - https://www.biorxiv.org/content/10.64898/2026.09.10.750654v1
AB - Learning novel behaviors requires balancing previously learned actions with the ability to flexibly adapt to changing reward contingencies. This trade-off is well documented in the division of labor between dorsolateral striatum (DLS), which promotes selection of cached, history-dependent actions, and dorsomedial striatum (DMS), which supports flexible learning as reward contingencies change. Here, we propose policy regularization as a general computational principle for understanding this division. Capacity-limited agents face a fundamental trade-off between maximizing reward and minimizing the cost of deviating from a default policy that caches frequently used action transitions. We formalize this trade-off as a KL-regularized reward objective in which a flexible controller (DMS) incurs a cost for diverging from a history-dependent default (DLS). The resulting optimal policy is a combination of a reward-driven action value, continuously updated by DMS, and a default policy conditioned on action history, cached by DLS. In practice, DLS consolidates the action transitions shaped by DMS's reward-driven value learning, so DLS preserves behaviors that were once optimal even when reward contingencies change, producing robust but inflexible action selection and effectively "regularizing" DMS-driven learning. We show that a single model with one shared parameter set and consistent lesion rules provides a unifying explanation for the functional organization of the striatum, reproducing canonical DLS-DMS dissociations in outcome devaluation, serial spatial reversal, skilled action sequencing, and motor sequence execution tasks.
ER -