Risk-sensitive safe reinforcement learning for joint cyber insurance and cybersecurity investment planning in cyber-physical power systems
Abstract
Cyber insurance and cybersecurity investment are tightly coupled in smart grids because cyberattacks can produce physical outages, financial losses, regulatory exposure, and insurance claims through the same operating event. This study develops a risk-sensitive safe reinforcement learning framework for joint cyber insurance and defense investment planning in cyber-physical power systems. The formulation treats the utility as a sequential decision maker that observes physical grid states, cyber exposure, insurance contract status, posterior threat beliefs, and budget conditions, and then selects coverage, deductible, hardening, monitoring, redundancy, backup-control, and incident-response actions. The framework combines a constrained Markov decision process, conditional value-at-risk loss control, an optimal power flow (OPF) safety shield, and a Benders-style decomposition between long-term policy learning and short-term security correction. The mathematical presentation separates model-specific cyber-physical-insurance equations from standard reinforcement-learning components, so that the novel coupling among insurance, defense, physical feasibility, and tail-risk control remains explicit. A simulation benchmark on IEEE test systems evaluates expected cost, 95% CVaR, physical-security violations, shield corrections, policy sensitivity, and belief-update degradation. Under the extreme-threat case, the safe-RL policy reduces 95% CVaR from 130.63 MUSD under the no-insurance stress case to 68.21 MUSD, lowers total cost from 36.8 MUSD to 17.1 MUSD, and reduces total security violations to four events per 1000 evaluation episodes. Across five random seeds, the total-cost and tail-risk results remain stable, with a mean total cost of 17.16 MUSD and a mean extreme-threat CVaR of 68.34 MUSD. The results show that cyber insurance is most effective when premiums are priced from verified defense posture and coordinated with physical safety constraints rather than treated as a substitute for cybersecurity investment.
Related
- Risk-aware hierarchical reinforcement learning for investment and operations in cross-border energy supply chains under the belt and road initiative
- Entropy-Regularized Reinforcement Learning for the Regime-Switching Consumption-Investment Problem
- Hierarchical reinforcement learning with model predictive elevator control for risk-sensitive multi-robot transport scheduling in hospitals
- Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations
- Provably Efficient Risk-Sensitive Offline Reinforcement Learning via Optimized Certainty Equivalents
BibTeX
@article{alhazmi2026risksensitive,
title = {{Risk-sensitive safe reinforcement learning for joint cyber insurance and cybersecurity investment planning in cyber-physical power systems}},
author = {Mohannad Alhazmi and Xiangge Huang},
journal = {Scientific Reports},
year = {2026},
month = nov,
doi = {10.1038/s41598-026-64656-9},
url = {https://www.nature.com/articles/s41598-026-64656-9},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Risk-sensitive safe reinforcement learning for joint cyber insurance and cybersecurity investment planning in cyber-physical power systems
AU - Mohannad Alhazmi
AU - Xiangge Huang
PY - 2026
DA - 2026/11/03
JO - Scientific Reports
DO - 10.1038/s41598-026-64656-9
UR - https://www.nature.com/articles/s41598-026-64656-9
AB - Cyber insurance and cybersecurity investment are tightly coupled in smart grids because cyberattacks can produce physical outages, financial losses, regulatory exposure, and insurance claims through the same operating event. This study develops a risk-sensitive safe reinforcement learning framework for joint cyber insurance and defense investment planning in cyber-physical power systems. The formulation treats the utility as a sequential decision maker that observes physical grid states, cyber exposure, insurance contract status, posterior threat beliefs, and budget conditions, and then selects coverage, deductible, hardening, monitoring, redundancy, backup-control, and incident-response actions. The framework combines a constrained Markov decision process, conditional value-at-risk loss control, an optimal power flow (OPF) safety shield, and a Benders-style decomposition between long-term policy learning and short-term security correction. The mathematical presentation separates model-specific cyber-physical-insurance equations from standard reinforcement-learning components, so that the novel coupling among insurance, defense, physical feasibility, and tail-risk control remains explicit. A simulation benchmark on IEEE test systems evaluates expected cost, 95% CVaR, physical-security violations, shield corrections, policy sensitivity, and belief-update degradation. Under the extreme-threat case, the safe-RL policy reduces 95% CVaR from 130.63 MUSD under the no-insurance stress case to 68.21 MUSD, lowers total cost from 36.8 MUSD to 17.1 MUSD, and reduces total security violations to four events per 1000 evaluation episodes. Across five random seeds, the total-cost and tail-risk results remain stable, with a mean total cost of 17.16 MUSD and a mean extreme-threat CVaR of 68.34 MUSD. The results show that cyber insurance is most effective when premiums are priced from verified defense posture and coordinated with physical safety constraints rather than treated as a substitute for cybersecurity investment.
SN - 2045-2322
ER -