Stochastic Decision Policy Is Optimal Under Costly Control
- 1Chinese Academy of Science
Abstract
Human and animal decisions are often noisy. This variability is usually attributed to noise that corrupts the decision evidence, a defect that an ideal decision maker would eliminate. Here we ask whether stochastic choice can instead be optimal once the cost of implementing prescribed policies is counted. We treat the decision maker as an ideal observer that maximizes expected gain but pays costs to sharpen its policy away from a random default. Using sequential sampling, a setting in which the optimal policy is precisely defined, we show that this trade-off makes a stochastic policy optimal, replacing the hard decision boundaries of classical theory with soft stopping zones whose softness is set by the price of control. This soft policy aligns with the strategy used in cutting-edge artificial learning agents for robustness. Noise that corrupts the agent's belief, an important confound of noisy behavior, produces superficially similar yet distinctive signatures in behavioral policies. Leveraging this, we recovered the policies of two macaques with low-dimensional recurrent neural networks and found that their behavior carries both a stochastic policy and belief noise. By separating variability in constructing a decision belief from stochasticity in selecting actions, these results establish that stochastic decision policies can themselves be optimal, offering a common principle for understanding behavioral variability in biological and artificial decision makers.
Related
- Optimal long-run control of endemic infections: bang-bang threshold policies in a stochastic SIS model
- Beyond Scalar Choice: Foraging as a Policy Framework for Decision Neuroscience
- Stochastic Procrastination and Optimal Task Division
- Decision-making pathways for adopting, implementing, and scaling gamified physical activity interventions in municipal policy and practice in the Netherlands
- A Bayesian framework reveals heterogeneous and stochastic decision-making in NK cell cytotoxicity
BibTeX
@misc{li2026stochastic,
title = {{Stochastic Decision Policy Is Optimal Under Costly Control}},
author = {J. Li and K. Louie and A. Vittala and M. Mattar and P. Glimcher and B. Shen},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.06.757200},
url = {https://www.biorxiv.org/content/10.64898/2026.10.06.757200v1},
note = {CC BY-ND},
}
RIS
TY - UNPB
TI - Stochastic Decision Policy Is Optimal Under Costly Control
AU - J. Li
AU - K. Louie
AU - A. Vittala
AU - M. Mattar
AU - P. Glimcher
AU - B. Shen
PY - 2026
DA - 2026/10/06
PB - bioRxiv
DO - 10.64898/2026.10.06.757200
UR - https://www.biorxiv.org/content/10.64898/2026.10.06.757200v1
AB - Human and animal decisions are often noisy. This variability is usually attributed to noise that corrupts the decision evidence, a defect that an ideal decision maker would eliminate. Here we ask whether stochastic choice can instead be optimal once the cost of implementing prescribed policies is counted. We treat the decision maker as an ideal observer that maximizes expected gain but pays costs to sharpen its policy away from a random default. Using sequential sampling, a setting in which the optimal policy is precisely defined, we show that this trade-off makes a stochastic policy optimal, replacing the hard decision boundaries of classical theory with soft stopping zones whose softness is set by the price of control. This soft policy aligns with the strategy used in cutting-edge artificial learning agents for robustness. Noise that corrupts the agent's belief, an important confound of noisy behavior, produces superficially similar yet distinctive signatures in behavioral policies. Leveraging this, we recovered the policies of two macaques with low-dimensional recurrent neural networks and found that their behavior carries both a stochastic policy and belief noise. By separating variability in constructing a decision belief from stochasticity in selecting actions, these results establish that stochastic decision policies can themselves be optimal, offering a common principle for understanding behavioral variability in biological and artificial decision makers.
ER -