Research on Intelligent Control Technology for Cooperative Game Implementation in Source-Grid-Load-Storage Systems Based on Reinforcement Learning
DOI:
https://doi.org/10.13052/dgaej2156-3306.4126Keywords:
Grid-load-storage coordination, deep reinforcement learning, proximal policy optimization (PPO), cooperative games, uncertainty scheduling, intraday rolling optimization, renewable energy integrationAbstract
As the penetration rate of renewable energy sources such as wind and solar power continues to rise, coordinated control among multiple entities including generation, transmission, load and storage has become crucial for ensuring the economic efficiency and security of power systems. However, the uncertainty of renewable energy output, the multi-period coupling characteristics of flexible resources, and the inconsistency of benefits among entities make it challenging for traditional optimization methods to simultaneously address real-time responsiveness, robustness, and fairness. To address this, this paper proposes an intelligent control method for power generation, grid, load, and storage that integrates Proximal Policy Optimization (PPO) with cooperative game theory. First, a Markov decision model suitable for multi-source, multi-load systems is constructed. The continuous action space of flexible resources enables coordinated control of thermal power, energy storage, and adjustable loads. Subsequently, a penalty for deviation from cooperative payoffs is embedded in the reward function, ensuring that policy optimization simultaneously satisfies overall economic efficiency and inter-agent profit coordination requirements. Multi-scenario simulations on IEEE 33-node and IEEE 30-node systems demonstrate that this method achieves rapid and stable convergence, significantly reduces operational costs, smooths power fluctuations, and maintains sustainable SOC for energy storage. Compared to conventional methods, it exhibits stronger robustness and higher cooperative incentive effects under uncertain conditions.
Downloads
References
Yang, H., Wei, X., Yang, Y., Li, Z., and Shen, S. (2024, September). Intelligent scheduling of source-grid-load-storage resources based on q-learning algorithm. In 2024 4th International Conference on Energy, Power and Electrical Engineering (EPEE) (pp. 1247–1251). IEEE.
Ma, Y., Zhao, J., Yang, K., and Zeng, H. (2024, December). Research on Deep Learning-Based Collaborative Optimization Planning Strategy for Source-GridLoad-Storage Coordination. In 2024 4th International Conference on Intelligent Power and Systems (ICIPS) (pp. 633–636). IEEE.
Zhou, D., Li, F., Zeng, P., Du, Z., and Jiao, K. (2024). Research on Key Technologies of “Source-Grid-Load-Storage” Collaborative Regulation and Interaction in the Park Based on Digital Twin Technology. In Mechatronics and Automation Technology (pp. 59–68). IOS Press.
Zang, T., Wang, S., Wang, Z., Li, C., Liu, Y., Xiao, Y., and Zhou, B. (2024). Integrated planning and operation dispatching of source–grid–load–storage in a new power system: A coupled socio–cyber–physical perspective. Energies, 17(12), 3013.
Cui, Z., Mo, C., Luo, Q., and Zhou, C. (2024). Integrated energy trading algorithm for source-grid-load-storage energy system based on distributed machine learning. Energy Informatics, 7(1), 149.
Manas, M. (2018). Optimization of distributed generation-based hybrid renewable energy system for a DC micro-grid using particle swarm optimization. Distributed Generation & Alternative Energy Journal, 33(4), 7–25.
Meydani, A., Shahinzadeh, H., Ramezani, A., Moazzami, M., Nafisi, H., and Askarian-Abyaneh, H. (2024, February). Comprehensive review of artificial intelligence applications in smart grid operations. In 2024 9th International Conference on Technology and Energy Management (ICTEM) (pp. 1–13). IEEE.
Bai, F., and Zhang, K. (2025). The research progress on distributed generation and microgrid system stability: Progress, challenges, and frontier directions. Advances in Resources Research, 5(4), 2366–2408.
Zhang, C., Roh, B. H., and Shan, G. (2024, June). Federated anomaly detection. In 2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks-Supplemental Volume (DSN-S) (pp. 148–149). IEEE.
Li, Y., Zhou, T., and Jin, G. (2025). The research progress on multi-energy system integrated stability: Modeling methods, stability analysis, and coordinated control strategies. Advances in Resources Research, 5(4), 2409–2453.
Kumar, P. M., Kamruzzaman, M. M., Alfurhood, B. S., Hossain, B., Nagarajan, H., and Sitaraman, S. R. (2024). Balanced performance merit on wind and solar energy contact with clean environment enrichment. IEEE Journal of the Electron Devices Society, 12, 808–823.
Zhou, Y. (2024). Energy-Sharing Economy with Renewable Integration and Management in Communities – a State-of-the-Art Review. Advanced Energy and Sustainability Research, 5(12), 2400214.
Ruiyi, C., Rui, G., Qisheng, L., Xingwang, J., Ling, S., Guanbao, Z., and Jia, H. (2025, April). Analysis of an IoT-Based Multi-Agent Relay Protection System and Cooperative Control Strategies. In 2025 10th Asia Conference on Power and Electrical Engineering (ACPEE) (pp. 1093–1097). IEEE.
Li, Y., Li, Y., Song, Z., and Ma, Z. (2025). The research on resilient power system construction for high-proportion renewable energy: Responsiveness enhancement and intelligent development pathways. Advances in Resources Research, 5(3), 1381–1421.
Yaoxin, W., Chun, Z., Lin, J., Houming, S., Qirui, W., and Zhuo, W. (2025, March). A Comprehensive Review on the Application of Artificial Intelligence in the Planning of the New-Type Power Systems. In 2025 7th Asia Energy and Electrical Engineering Symposium (AEEES) (pp. 1373–1382). IEEE.
Miao, L., Zhou, N., Ma, J., Liu, H., Zhao, J., Wei, X., and Yin, J. (2025). Current Status, Challenges and Future Perspectives of Operation Optimization, Power Prediction and Virtual Synchronous Generator of Microgrids: A Comprehensive Review. Energies, 18(13), 3557.
Deepak, K., Mandal, R. K., and Verma, V. (2023). Power Quality Improvement of a Hybrid Renewable Energy Sources Based Standalone System Using Neuro-Fuzzy Controllers. Distributed Generation & Alternative Energy Journal, 1815–1838.
Gao, X., Zhang, J., Sun, H., Liang, Y., Wei, L., Yan, C., and Xie, Y. (2024). A review of voltage control studies on low voltage distribution networks containing high penetration distributed photovoltaics. Energies, 17(13), 3058.
Mulevu, C. M., Okeyo, G. O., and Wafula, J. M. (2025). Data stream mining techniques for real-time monitoring and control of smart power grids in Kenya: challenges and opportunities. Discover Internet of Things, 5(1), 51.
Wang, K., Huang, Y., Liu, Y., Huang, T., and Zang, S. (2025). A Review of Optimization Scheduling for Active Distribution Networks with High-Penetration Distributed Generation Access. Energies, 18(15), 4119.
Zhu, L., Wu, X., Wang, Z., Li, Y., Song, L., and Yang, Y. (2025). A Study on the Environmental and Economic Benefits of Flexible Resources in Green Power Trading Markets Based on Cooperative Game Theory: A Case Study of China. Energies, 18(17), 4490.
He, Y., Liu, T., Wang, Z., Ren, Q., and Chen, A. (2025). Adaptive and Collaborative Hierarchical Optimization Strategies for a Multi-Microgrid System Considering EV and Storage. World Electric Vehicle Journal, 16(7), 363.
Gu, S., Qian, K., and Yang, Y. (2025). Optimization of Electric Vehicle Charging and Discharging Strategies Considering Battery Health State: A Safe Reinforcement Learning Approach. World Electric Vehicle Journal, 16(5), 286.
Zhang, Z., Li, Y., Lai, D., Zhou, N., Zhan, Q., and Wang, W. (2025). Quantifying Weather’s Share in Dynamic Grid Emission Factors via SHAP: A Multi-Timescale Attribution Framework. Processes, 13(11), 3393.
Wang, D., and Wu, S. (2025). Accurate Modeling of Carbon Emissions Under Urban Energy Consumption. Distributed Generation & Alternative Energy Journal, 1259–1280.
Liu, C., Ye, C., Xu, J., Wang, Z., Wei, S., Yang, J., … and Liu, F. (2025, August). An Optimal Configuration Model For Photovoltaic And Energy Storage Access To The Distribution Network Considering Demand Response And EV Load. In 2025 3rd International Conference on Power, Grid and Energy Storage (pp. 283–288). IEEE.
Liu, H., Yu, J., Wang, M., Zhou, M., and Wang, D. (2025). Source-load joint forecasting of office building electro-thermal integrated energy systems based on multi-task learning and dynamic optimal integration. International Journal of Green Energy, 1–19.
Kulkarni, S. H., Jirage, B. J., and Anil, T. R. (2017). Alternative energy options for India – a multi-criteria decision analysis to rank energy alternatives using analytic hierarchy process and fuzzy logic with an emphasis to distributed generation. Distributed Generation & Alternative Energy Journal, 32(2), 29–55.
Jiang, H., Tong, B., Yao, Z., and Zhao, Y. (2025). Hierarchical Distributed Low-Carbon Economic Dispatch Strategy for Regional Integrated Energy System Based on ADMM. Energies, 18(17), 4638.
Shi, Z., Yan, L., Hu, Y., Wang, Y., Qin, W., Liang, Y., … and Zhang, Z. (2024). Optimization of operation strategy of multi-islanding microgrid based on double-layer objective. Energies, 17(18), 4614.
