Distributed Energy Storage Scheduling Optimization Based on Improved Multi-agent Deep Deterministic Policy Gradient Algorithm
DOI:
https://doi.org/10.13052/dgaej2156-3306.41510Keywords:
Distributed energy storage dispatch, DDPG, Electric-hydrogen hybrid microgrid, MAPPO, Multi-microgrid collaborationAbstract
The core of current energy storage scheduling optimization is to achieve multi-timescale collaborative decision-making through intelligent algorithms to improve system economy and reliability, and accelerate its evolution towards market-oriented and large-scale applications. This study is designed to develop a scenario-adaptive intelligent scheduling system. It skips the need for accurate long-term future time-series predictions, and directly leverages real-time observable information at the current decision point, such as renewable energy output, power load, electricity price and energy storage status, to complete dynamic optimal scheduling. For a single independent microgrid, an electric-hydrogen hybrid architecture is constructed, and an improved deep deterministic policy gradient algorithm with attenuated random noise is proposed. The scheduling strategy is optimized through online interaction with the target network and a soft update mechanism. For multiple interconnected microgrids, a centralized training and decentralized execution framework is adopted to achieve multi-agent collaborative optimization and autonomous decision-making. The results show that in a single microgrid scenario, the research method achieves a renewable energy utilization efficiency of 98.77% and a average operating cost of 0.381 yuan/kWh; in a multi-microgrid scenario, the average operating cost is 0.389 yuan/kWh. The research indicates that the two types of algorithms are respectively adapted to single-microgrid internal optimization and multi-microgrid collaborative scheduling, providing scenario-based solutions for distributed energy storage optimization.
Downloads
References
Yin Z, Zhang X, Luo W, Ruan S. Scalable Power Dispatch Automation Methods Based on Artificial Intelligence in Distributed Energy Systems. Distributed Generation & Alternative Energy Journal, 2026, 41(3): 791–814. DOI: 10.13052/dgaej2156-3306.41310.
Zhong L S, Zeng Z L, Huang Z K, Shi X W, Bie Y M. Joint optimization of electric bus charging and energy storage system scheduling. Frontiers of Engineering Management, 2024, 11(4): 676–696. DOI: 10.1007/s42524-024-3102-2.
Chen J, Hu N. Distributed Optimization Model for Economic Dispatch of Smart Grid. Distributed Generation & Alternative Energy Journal, 2025, 40(3): 457–480. DOI: 10.13052/dgaej2156-3306.4031.
Zhou Z, Bian J Y, Yu Z Y. Incremental cost analysis model of distribution network based on economic dispatch of distributed new-energy storage system. Clean Energy, 2024, 8(2): 89–103. DOI: 10.1093/ce/zkae007.
Huang Y, Pan L, Chen J, Pang Y, Shi F, Chen L, et al. Robust optimal operation scheduling method for integrated energy system considering flexible energy storage and mixed hydrogen natural gas. Energy Sources, Part A: Recovery, Utilization, and Environmental Effects, 2024, 46(1): 3476–3498. DOI: 10.1080/15567036.2024.2319723.
Lu Z X, Xu X Y, Yan Z, Shahidehpour M, Sun W Q, Han D. Distributionally robust chance constrained optimization method for risk-based routing and scheduling of shared mobile energy storage system with variable renewable energy. IEEE Transactions on Sustainable Energy, 2024, 15(4): 2594–2608. DOI: 10.1109/TSTE.2024.3429310.
Dong L, Huang Y, Xu X, Zhang Z Y, Liu J Y, Pan L, Hu W H. Research on priority scheduling strategy for smoothing power fluctuations of microgrid tie-lines based on PER-DDPG algorithm. IET Generation, Transmission & Distribution, 2024, 18(20): 3221–3233. DOI: 10.1049/gtd2.13267.
Krishna R, Hemamalini S. Long short-term memory-based forecasting of uncertain parameters in an islanded hybrid microgrid and its energy management using improved grey wolf optimization algorithm. IET Renewable Power Generation, 2024, 18(16): 3640–3658. DOI: 10.1049/rpg2.13115.
Todakar K M, Gupta P P, Kalkhambkar V, Sharma K C. Optimal scheduling of battery energy storage train and renewable power generation. Electrical Engineering, 2024, 106(5): 6477–6493. DOI: 10.1007/s00202-024-02385-w.
Yan D M, Wang H K, Gao Y J, Tian S J, Zhang H. Based on improved crayfish optimization algorithm cooperative optimal scheduling of multi-microgrid system. Scientific Reports, 2024, 14(1): 1–18. DOI: 10.1038/s41598-024-76041-5.
Sui Q, Zhang J Y, Sun L, Liang J, Wei F R, Lin X N. Optimal scheduling of mobile energy storage capable of variable speed energy transmission. IEEE Transactions on Smart Grid, 2024, 15(3): 2710–2722. DOI: 10.1109/TSG.2023.3329294.
Sun H N, Zhang B C, Liu N. Imitation learning-based online optimal scheduling for microgrids: An approach integrating input clustering and output classification. IET Renewable Power Generation, 2024, 18(15): 3161–3172. DOI: 10.1049/rpg2.12909.
Tian J J, Qian Y, Zhao F, Mo S L, Xiao H X, Zhu X T, et al. Energy-saving optimal scheduling under multi-mode “source-network-load-storage” combined system in metro station based on modified Gray Wolf Algorithm. Archives of Electrical Engineering, 2024, 73(1): 121–143. DOI: 10.24425/aee.2024.148861.
Xue L, Niu T, Ge H, Zhang J, Xue Y, Fang S, et al. A joint distributed optimization framework for voltage control and emergency energy storage vehicle scheduling in community distribution networks. IEEE Transactions on Industry Applications, 2024, 60(4): 5317–5330. DOI: 10.1109/TIA.2024.3384474.
Qian T, Ming W L, Shao C C, Hu Q R, Wang X L, Wu J Z, et al. An edge intelligence-based framework for online scheduling of soft open points with energy storage. IEEE Transactions on Smart Grid, 2024, 15(3): 2934–2945. DOI: 10.1109/TSG.2023.3330990.
Ghaffarpour R, Zamanian S. A robust bi-level programming approach for optimal planning an off-grid zero-energy complex. IET Renewable Power Generation, 2024, 18(14): 2394–2415. DOI: 10.1049/rpg2.13083.
Zhang H X. Application of day-ahead optimal scheduling model based on multi energy micro-grids with uncertainty in wind and solar energy and energy storage station. International Journal of Renewable Energy Development, 2024, 13(5): 873–883. DOI: 10.61435/ijred.2024.60218.
Tang Y, Zhai Q Z, Zhao J X. Multi-stage robust economic dispatch with virtual energy storage and renewables based on a single level model. IEEE Transactions on Automation Science and Engineering, 2024, 21(4): 5490–5502. DOI: 10.1109/TASE.2023.3312379.
Kang K, Zhang Y L, Miu Y J, Gao Q, Chen K W, Zeng Z H. Study on master-slave game optimization operation of integrated energy microgrid considering PV output uncertainty and shared energy storage. Journal of Advanced Computational Intelligence and Intelligent Informatics, 2024, 28(3): 528–540. DOI: 10.20965/jaciii.2024.p0528.
Shokri M, Niknam T, Mohammadi M, Dehghani M, Siano P, Ouahada K, et al. A novel stochastic framework for optimal scheduling of smart cities as an energy hub. IET Generation, Transmission & Distribution, 2024, 18(14): 2421–2434. DOI: 10.1049/gtd2.13202.
Waqas M, Naseem A. Artificial intelligence in sustainable industrial transformation: A comparative study of Industry 4.0 and Industry 5.0. FinTech Sustain. Innov, 2025, 1: A2. DOI: 10.47852/bonviewFSI52025321.
Kumar A, Dhillon J S. Emended Harris Hawk optimizer for mixed energy generation scheduling problem. Electric Power Components and Systems, 2024, 52(8): 1235–1268. DOI: 10.1080/15325008.2023.2239224.
