Multi-layer collaborative control method for user-side resources based on deep deterministic policy gradient algorithm
- 1NARI Group Corporation (State Grid Electric Power Research Institute)
- 2NARI-TECH Nanjing Control Systems Co., Ltd.
Abstract
User-side resources (USRs), including flexible loads and distributed storage, are increasingly integrated into grid regulation. However, their inherent heterogeneity, response uncertainty, and geographically dispersed deployment pose significant challenges to unified and scalable control. Existing regulation strategies often struggle to simultaneously capture multi-layer coordination requirements and the dynamic flexibility boundaries of heterogeneous resources. To address this issue, this paper proposes a multi-layer collaborative control framework in which upper-layer decision-making is explicitly coupled with lower-layer flexibility intervals. Specifically, the upper-layer controller allocates regulation tasks across aggregated USRs, while the lower layer decomposes these tasks into resource-level actions constrained by dynamically evolving flexible intervals. The overall decision process is formulated as a Markov decision process (MDP), and a Deep Deterministic Policy Gradient (DDPG) algorithm is employed to learn optimal control policies for continuous decision spaces inherent in USR regulation. To ensure effective learning, a composite reward function is designed to jointly capture operating cost, regulation deviation, and constraint violation penalties, enabling the agent to balance economic efficiency and control accuracy. Extensive simulation studies, including convergence analysis, baseline comparisons, computational efficiency evaluation, ablation studies, and hyperparameter sensitivity analysis, are conducted to validate the proposed approach. The results demonstrate that the proposed framework significantly improves peak-shaving performance and operational economy, while achieving stable convergence and strong adaptability in complex multi-layer regulation scenarios.
Related
- Whole genome sequencing and tuberculosis surveillance and control: a critical view from resources limited settings
- Dynamic analysis and chaos control of an environmental population model incorporating the wind flow in a stochastic and deterministic perspective
- A genetic algorithm–optimized EWMA control chart integrating machine learning-based risk prediction for multistage thyroid cancer treatment monitoring
- Fragmented and Underutilized: Exploring Role Variation and Ambiguity for Research Project Managers in Collaborative Team Science
- Where did it go? How Information Management Practices Impact Rigor and Reproducibility in Collaborative Clinical and Translational Research
BibTeX
@article{zheng2026multilayer,
title = {{Multi-layer collaborative control method for user-side resources based on deep deterministic policy gradient algorithm}},
author = {Tao Zheng and Yufeng Yang and Xianliang Teng and Jian Geng and Yulong Jin and Zhicheng Zhou},
journal = {Scientific Reports},
year = {2026},
month = jul,
doi = {10.1038/s41598-026-63158-y},
url = {https://doi.org/10.1038/s41598-026-63158-y},
note = {CC BY},
}
RIS
TY - JOUR
TI - Multi-layer collaborative control method for user-side resources based on deep deterministic policy gradient algorithm
AU - Tao Zheng
AU - Yufeng Yang
AU - Xianliang Teng
AU - Jian Geng
AU - Yulong Jin
AU - Zhicheng Zhou
PY - 2026
DA - 2026/07/01
JO - Scientific Reports
DO - 10.1038/s41598-026-63158-y
UR - https://doi.org/10.1038/s41598-026-63158-y
AB - User-side resources (USRs), including flexible loads and distributed storage, are increasingly integrated into grid regulation. However, their inherent heterogeneity, response uncertainty, and geographically dispersed deployment pose significant challenges to unified and scalable control. Existing regulation strategies often struggle to simultaneously capture multi-layer coordination requirements and the dynamic flexibility boundaries of heterogeneous resources. To address this issue, this paper proposes a multi-layer collaborative control framework in which upper-layer decision-making is explicitly coupled with lower-layer flexibility intervals. Specifically, the upper-layer controller allocates regulation tasks across aggregated USRs, while the lower layer decomposes these tasks into resource-level actions constrained by dynamically evolving flexible intervals. The overall decision process is formulated as a Markov decision process (MDP), and a Deep Deterministic Policy Gradient (DDPG) algorithm is employed to learn optimal control policies for continuous decision spaces inherent in USR regulation. To ensure effective learning, a composite reward function is designed to jointly capture operating cost, regulation deviation, and constraint violation penalties, enabling the agent to balance economic efficiency and control accuracy. Extensive simulation studies, including convergence analysis, baseline comparisons, computational efficiency evaluation, ablation studies, and hyperparameter sensitivity analysis, are conducted to validate the proposed approach. The results demonstrate that the proposed framework significantly improves peak-shaving performance and operational economy, while achieving stable convergence and strong adaptability in complex multi-layer regulation scenarios.
SN - 2045-2322
ER -