Multi-agent Reinforcement Learning-based Basic Data Collection and Dynamic Information Evaluation for Power Station Primary Frequency
DOI:
https://doi.org/10.13052/dgaej2156-3306.4133Keywords:
Primary frequency regulation, renewable integration, multi-energy system coordination, data-driven grid flexibility, low-carbon power system operation, real-time performance assessment, industrial-scale frequency controlAbstract
Primary Frequency Regulation (PFR) of power stations is faced with challenges such as intensified frequency dynamic fluctuations, complex coordinated control of multiple power sources, and unbalanced operating economy. Traditional data collection methods are difficult to meet the demand for precise control of PFR. This study intends to establish a multi-dimensional data collection system to improve the accuracy of primary frequency regulation dynamic information evaluation and strategy optimization effects. It first designs a multi-source acquisition framework covering grid-side frequency indicators and power station-side equipment operation data, and combines time series interpolation and outlier detection for data preprocessing. Then, a dynamic information evaluation model based on multi-agent proximal strategy optimization is built to achieve multi-power collaborative evaluation through centralized training and decentralized execution mode. Finally, an improved particle swarm optimization algorithm is used to optimize the frequency regulation strategy. Based on the on-site measured data of a provincial-level integrated energy power station (including 4 types of power sources and continuous operation for 30 days), the research results show that the data integrity of the proposed data collection system was improved to 98.7%, and the frequency deviation prediction error of the dynamic evaluation model was controlled within ±0.02 Hz. The optimization strategy increased the lowest point of frequency by 0.03–0.05 Hz, and reduced the total cost of frequency regulation by 12.3%. The study provides accurate data support and efficient control solutions for PFR of power stations, which has important practical significance for improving the frequency stability and operating economy of the power system.
Downloads
References
Zhang J, Wang Y, Zhou G, Wang L, Li B, Li K. Integrating physical and data-driven system frequency response modelling for wind-PV-thermal power systems[J]. IEEE Transactions on Power Systems, 2023, 39(1): 217–228. DOI:10.1109/TPWRS.2023.3242832.
Zhang Z, Kou P, Zhang Y, Liang D. Coordinated predictive control of offshore DC collection grid and wind turbines for frequency response: A scheme without secondary frequency drop[J]. IEEE Transactions on Sustainable Energy, 2023, 14(3): 1488–1503. DOI:10.1109/TSTE.2023.3236721.
Demirci H E, Jalbi S, Bhattacharya S. Liquefaction effects on the fundamental frequency of monopile supported offshore wind turbines (OWTs)[J]. Bulletin of Earthquake Engineering, 2022, 20(7): 3359–3384. DOI:10.1007/s10518-022-01360-9.
Pourbeik P, Sanchez-Gasca J J, Senthil J, Weber J, Zadkhast P, Ramasubramanian D, et al. A generic model for inertia-based fast frequency response of wind turbines and other positive-sequence dynamic models for renewable energy systems[J]. IEEE Transactions on Energy Conversion, 2023, 39(1): 425–434. DOI:10.1109/TEC.2023.3315058.
Yang X, Yang L, Xiao X, Wang Y. A novel detection method for supersynchronous resonance from synchrophasor data[J]. IEEE Transactions on Power Systems, 2022, 38(4): 3694–3706. DOI:10.1109/TPWRS.2022.3200593.
Wei M, Shi F, Zhang H, Chen W. Wideband synchronous measurement-based detection and location of high impedance fault for resonant distribution systems with integration of DERs[J]. IEEE Transactions on Smart Grid, 2022, 14(2): 1117–1134. DOI:10.1109/TSG.2022.3199781.
Liang Y, Zhao X, Sun L. A multiagent reinforcement learning approach for wind farm frequency control[J]. IEEE Transactions on Industrial Informatics, 2022, 19(2): 1725–1734. DOI:10.1109/TII.2022.3182328.
Yang F, Huang D H, Li D, Lin S, Muyeen S M, Zhai H. Data-driven load frequency control based on multi-agent reinforcement learning with attention mechanism[J]. IEEE Transactions on Power Systems, 2022, 38(6): 5560–5569. DOI:10.1109/TPWRS.2022.3223255.
Li J, Yang S, Yu T. Data-driven cooperative load frequency control method for microgrids using effective exploration-distributed multi-agent deep reinforcement learning[J]. IET renewable power generation, 2022, 16(4): 655–670. DOI:10.1049/rpg2.12323.
Wu Q, Li G, Liu M, Zhang Y, Yan J, Deguchi Y. The Enhancement of Primary Frequency Regulation Ability of Combined Water and Power Plant Based on Nuclear Energy: Dynamic Modelling and Control Strategy Optimization[J]. Energy, 2024, 313(Dev.30):133721.1–133721.16. DOI:10.1016/j.energy.2024.133721.
Wang B, Zhu S, Cai G, Yang D, Chen Z, Ma J, et al. Sparse measurement-based modelling low-order dynamics for primary frequency regulation[J]. IEEE Transactions on Power Systems, 2023, 39(1): 681–692. DOI:10.1109/tcomm.2023.3274145.
Kim J K, Kang J, Shim J W, Kim H, Shin J, Kang C, et al. Dynamic performance modeling and analysis of power grids with high levels of stochastic and power electronic interfaced resources[J]. Proceedings of the IEEE, 2023, 111(7): 854–872. DOI:10.1109/JPROC.2023.3284890.
Nguyen H T, Choi D H. Three-stage inverter-based peak shaving and Volt-VAR control in active distribution networks using online safe deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2022, 13(4): 3266–3277. DOI:10.1109/TSG.2022.3166192.
Shuai H, She B, Wang J, Li F. Safe reinforcement learning for grid-forming inverter based frequency regulation with stability guarantee[J]. Journal of Modern Power Systems and Clean Energy, 2024, 13(1): 79–86. DOI:10.35833/MPCE.2023.000882.
Zhang M, Guo G, Magnússon S, Pilawa-Podgurski R C, Xu Q. Data driven decentralized control of inverter based renewable energy sources using safe guaranteed multi-agent deep reinforcement learning[J]. IEEE Transactions on Sustainable Energy, 2023, 15(2): 1288–1299. DOI:10.1109/TSTE.2023.3341632.
Liu Q, Guo Y, Deng L, Liu H, Li D, Sun H, et al. Two-critic deep reinforcement learning for inverter-based volt-var control in active distribution networks[J]. IEEE Transactions on Sustainable Energy, 2024, 15(3): 1768–1781. DOI:10.1109/TSTE.2024.3376369.
Dev A, Mondal B, Verma V K, Kumar V. Teaching Learning Optimization-Based Sliding Mode Control for Frequency Regulation in Microgrid[J]. Electrical Engineering, 2024, 106(6):7009–7021. DOI:10.1007/s00202-024-02422-8.
Ma L, Hui H, Song Y. Data Valuation-Aware Coordinated Optimization of Power-Communication Coupled Networks Considering Hybrid Ancillary Services[J]. IEEE Transactions on Smart Grid, 2025, 16(1):568–581. DOI:10.1109/TSG.2024.3409814.
Baral K K, Nayak P C, Mohanty B, Barisal A K. Improved Frequency Regulation of Dual-Area Hybrid Power System with the Influence of Energy Storage Devices[J]. Electrical Engineering, 2025, 107(3):3511–3532. DOI:10.1007/s00202-024-02670-8.
Yang Q, Yan L, Chen X, Chen Y, Wen J. A distributed dynamic inertia-droop control strategy based on multi-agent deep reinforcement learning for multiple paralleled VSGs[J]. IEEE Transactions on Power Systems, 2022, 38(6): 5598–5612. DOI:10.1109/TPEL.2023.3286839.
Chen P, Liu S, Chen B, Yu L. Multi-agent reinforcement learning for decentralized resilient secondary control of energy storage systems against DoS attacks[J]. IEEE Transactions on Smart Grid, 2022, 13(3): 1739–1750. DOI:10.1109/TSG.2022.3142087.
Zhao Y, Zhong H, Lim C C. Safety-constrained multi-agent reinforcement learning for power quality control in distributed renewable energy networks[J]. Comput Mater Contin, 2024, 79(1): 449–471. DOI:10.32604/cmc.2024.048771.
Bounar A, Boubertakh H, Arbid M. A coordinated optimization strategy for energy management of hybrid electric vehicle fleets[J]. Journal of Control, Automation and Electrical Systems, 2025, 36(2): 300–311. DOI:10.1007/s40313-025-01148-7.
Qiu L. Multi-agent reinforcement learning for coordinated smart grid and building energy management across urban communities[J]. Computer Life, 2025, 13(3): 8–15. DOI:10.54097/3veq6255.
Doskenov B, Okuyelu O. Advancing production systems with online reinforcement learning: real-time monitoring, control, and optimization[J]. Current Journal of Applied Science and Technology, 2025, 44(2): 1–22. DOI:10.9734/cjast/2025/v44i24480.
