When Should a Human Take Back Control? Optimal Delegation under Turbulent AI Risk
Abstract
Deploying AI systems requires deciding when to delegate tasks and when humans should intervene to monitor and mitigate risk induced by AI operations. These decisions are challenging when failures cluster: a hallucination or harmful output can trigger further errors, creating periods of elevated risk. We introduce a continuous-time framework for learning adaptive human oversight under such turbulent AI risk. Existing oversight and delegation formulations condition on history but do not model incident clustering, or its suppression by supervision effort, jointly with the delegation decision and this study fixes this gap. The self-exciting dynamics capture how risk events increase the likelihood of subsequent events, making their timing and history central to decision-making. We formulate a stochastic control problem combining human actions, monitoring effort, and switching between human-AI-assisted operation and full AI delegation, balancing operational rewards against oversight costs, and cascading AI-failures and induced uncertainty. Human participation is an endogenous component of risk management: the policy determines both when oversight is needed and how much effort to allocate. We study a relaxed switching formulation and propose Hawkes-PPO, a policy-gradient method that uses a bank of exponential filters of observed incident times. In a synthetic environment it attains a higher risk-adjusted objective than either fixed regime and approaches an approximate full-information oracle. We illustrate our results with numerical simulations by examining how cascade risks influence intervention and delegation, connecting reinforcement learning with adaptive human oversight of AI systems. In particular, we illustrate the benefit of our switching strategy and Hawkes-PPO algorithm to monitor the project efficiently along time, reducing turbulent risks occurrences and costs.
Related
- Risk-Aware Output Control Should be the Focus of Machine Unlearning
- What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control
- Turnpike in optimal control and beyond: a survey
- Optimal Control of State Constrained Systems via Measure Relaxations and Polynomial Optimization
- Optimal Control of Vaccine Therapy in a Tumor-Immune-Vaccine Model for Pituitary Neuroendocrine Tumors
BibTeX
@misc{yan2026when,
title = {{When Should a Human Take Back Control? Optimal Delegation under Turbulent AI Risk}},
author = {Haoze Yan and Julien Roze and Ved Upadhyay and Unal Tatar and Thibaut Mastrolia},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.32083},
eprint = {2609.32083},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.32083},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - When Should a Human Take Back Control? Optimal Delegation under Turbulent AI Risk
AU - Haoze Yan
AU - Julien Roze
AU - Ved Upadhyay
AU - Unal Tatar
AU - Thibaut Mastrolia
PY - 2026
DA - 2026/09/25
PB - arXiv
DO - 10.48550/arxiv.2609.32083
UR - https://arxiv.org/abs/2609.32083
AB - Deploying AI systems requires deciding when to delegate tasks and when humans should intervene to monitor and mitigate risk induced by AI operations. These decisions are challenging when failures cluster: a hallucination or harmful output can trigger further errors, creating periods of elevated risk. We introduce a continuous-time framework for learning adaptive human oversight under such turbulent AI risk. Existing oversight and delegation formulations condition on history but do not model incident clustering, or its suppression by supervision effort, jointly with the delegation decision and this study fixes this gap. The self-exciting dynamics capture how risk events increase the likelihood of subsequent events, making their timing and history central to decision-making. We formulate a stochastic control problem combining human actions, monitoring effort, and switching between human-AI-assisted operation and full AI delegation, balancing operational rewards against oversight costs, and cascading AI-failures and induced uncertainty. Human participation is an endogenous component of risk management: the policy determines both when oversight is needed and how much effort to allocate. We study a relaxed switching formulation and propose Hawkes-PPO, a policy-gradient method that uses a bank of exponential filters of observed incident times. In a synthetic environment it attains a higher risk-adjusted objective than either fixed regime and approaches an approximate full-information oracle. We illustrate our results with numerical simulations by examining how cascade risks influence intervention and delegation, connecting reinforcement learning with adaptive human oversight of AI systems. In particular, we illustrate the benefit of our switching strategy and Hawkes-PPO algorithm to monitor the project efficiently along time, reducing turbulent risks occurrences and costs.
ER -