MPC-Guided Deep Reinforcement Learning for Real-Time Scheduling of Microgrid with Uncertainty
DOI:
https://doi.org/10.13052/dgaej2156-3306.4136Keywords:
Microgrid, real-time scheduling, model predictive control, deep reinforcement learning, deep deterministic policy gradientAbstract
Microgrid energy management plays a critical role in ensuring the secure and economical operation of microgrids. To address the uncertainty of renewable energy generation, this paper proposes an MPC-guided deep reinforcement learning (DRL)–based intraday scheduling strategy for microgrids. The proposed approach integrates the advantages of model predictive control (MPC) and DRL, where the optimization results of the MPC module are provided as environmental inputs to the DRL agent, and the DRL module interacts with the real microgrid environment to generate compensation actions. This framework not only mitigates the performance degradation caused by uncertainties in model-based methods, but also reduces the search space of DRL, thereby accelerating training convergence and suppressing policy fluctuations. Comparative simulations are conducted against standalone MPC and standalone DRL controllers. The results demonstrate that the proposed strategy can significantly reduce both operational security cost and economic cost, while effectively improving the utilization of renewable energy. Therefore, it provides an innovative solution for the microgrid scheduling problem.
Downloads
References
T. Appenzeller, 2025 Breakthrough of the Year, J. Science, 2025.
Li, G., Ji, S., Wang, M., et al. Research on Optimization and Scheduling Control Strategy of Renewable Energy Grid-Connection Based on Intelligent Control. J. Distributed Generation & Alternative Energy Journal, 41(01) 83–100.
Zhang Z, Guo L, Wu J, et al. Optimization of microgrid dispatching by integrating photovoltaic power generation forecast. J. Sustainability, 2025, 17(2) 648.
Xu, H. B., Yue B., and Zhang F. 2026. Optimal configuration of hybrid energy storage capacity for wind power fluctuation smoothing based on GSWOA-VMD. J. Distributed Generation and Alternative Energy. 41(01) 167–192..
Kumar K, Kwon S, Bae S. Deep reinforcement learning-based control strategy for integration of a hybrid energy storage system in microgrids. J. Journal of Energy Storage, 2025, 108: 114936
Shu, Y. K., W. Z. Bi, W. Dong, et al. Dueling Double Q-learning based Real-time Energy Dispatch in Grid-connected Microgrids. C. 2020 19th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES). IEEE, 2020.
Kuang, Y. J., D. H. Zhou, Z. W. Shen, et al. Proximal Policy Optimization-Based Reinforcement Learning Control of Single-Stage Multiport Inverter in Islanded Microgrids. J. IEEE Transactions on Industrial Electronics, 2026, 73(4) 5663–5674
Liu, Y. L., T. H. Qie, Y. Yu, et al. A Novel Integral Reinforcement Learning-Based Control Method Assisted by Twin Delayed Deep Deterministic Policy Gradient for Solid Oxide Fuel Cell in DC Microgrid. J. IEEE Transactions on Sustainable Energy, 2023, 14(1) 688–703.
Li Y, He S, Li Y, et al. Federated multiagent deep reinforcement learning approach via physics-informed reward for multi-microgrid energy management. J. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(5) 5902–5914.
Mo, S. Y., W. H. Chen, W. X. Zheng, et al. Distributed Hybrid Control for Heterogeneous Multiagent Systems With Variable Communication Delays and Its Application to DC Microgrids. J. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(12) 7501–7512.
X. Kong, X. Liu, L. Ma, and K. Y. Lee, Hierarchical distributed model predictive control of standalone wind/solar/battery power system. J. IEEE Trans. Syst. Man Cybern. Syst., vol. 49, no. 8: 1570–1581, Aug. 2019.
Z. Zhao, J. Xu, Y. Lei, C. Liu, et al. Robust dynamic dispatch strategy for multi-uncertainties integrated energy microgrids based on enhanced hierarchical model predictive control. J. Applied Energy, 2025, 381: 125141.
Wu L, Yin X, Pan L, Liu J. Economic model predictive control of integrated energy systems: A multi-time-scale framework. J. Applied Energy, 2022, 328: 120187
Zhu Z, Dong G, Lou Y, et al. MPC-guided deep reinforcement learning for optimal charging of lithium-ion battery with uncertainty. J. IEEE Transactions on Transportation Electrification, 2025.
Sun D, Jamshidnejad A, De Schutter B. A novel framework combining MPC and deep reinforcement learning with application to freeway traffic control. J. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(7) 6756–6769.
E.H. Sumiea, S.J. Abdulkadir, H.S. Alhussian, S.M. Al-Selwi, A. Alqushaibi, M.G. Ragab, S.M. Fati, Deep deterministic policy gradient algorithm: A systematic review, J. HELIYON, 10 (2024): 30697.
Ko, M. S., H. Zhu, and K. Hur. Deterministic and Probabilistic Forecasting of Wind Power Generation and Ramp Rate With Expectation-Implemented Deep Learning. J. IEEE Transactions on Sustainable Energy, 2026, 17(1) 338–350.
Lee S, Seon J, Sun Y G, et al. Novel architecture of energy management systems based on deep reinforcement learning in microgrid. J. IEEE Transactions on Smart Grid, 2024, 15(2) 1646–1658.
Wu J, Wei Z, Li W, et al. Battery thermal-and health-constrained energy management for hybrid electric bus based on soft actor-critic DRL Algorithm. J. IEEE Transactions on Industrial Informatics, 2021, 17(6) 3751–3761.
Zhang, Z. S., Y. Z. Sun, D. W. Gao, et al. A Versatile Probability Distribution Model for Wind Power Forecast Errors and Its Application in Economic Dispatch. J. IEEE Transactions on Power Systems, 2013, 28(3) 3114–3125.
Meijie Liu, Peng Qiu, Kai Wei. Research on Wind Speed Prediction of Wind Power System Based on GRU Deep Learning. Preprints of the 3rd IEEE Conference on Energy Internet and Energy System Integration. C. Changsha, China: IEEE, 2019.
Lu, C. Q., J. Li, G. D. Zhang, et al. A GRU-based short-term multi-energy loads forecast approach for integrated energy system. Proceedings of the 4th Asia Energy and Electrical Engineering Symposium. C. Chengdu, China, 2022.
Yidi, W., Xiaotian, M., Mengyu, L., et al. A New Representative Power Station Selection Method in Distributed Photovoltaic Cluster Power Forecasting. J. Distributed Generation & Alternative Energy Journal, 40(05-06): 1183–1208.
