Abstract
Cyber-physical systems (CPS), such as power plants and critical infrastructure, face growing safety and security risks due to increased interconnectivity and automation. To address these challenges, we propose a framework that combines deep reinforcement learning (DRL) and Bayesian probability to model and assess vulnerabilities and attack paths. Leveraging Deep Q-Learning Networks (DQN) and cumulative probability, our method improves the identification and prioritization of effective attack paths. We also introduce the attack-fault tree (AFT) model to evaluate interactions between safety and security events. Using Uppaal SMC and statistical timed automata, we simulate dynamic scenarios and generate probabilistic estimates of metrics such as attack cost, duration, and impact. By analyzing both current and hypothetical scenarios, our approach supports informed, adaptive defense strategies for resilient CPS.
References
Eurelectric, “Cybersecurity in the power sector,” Oct. 2023. [Online]. Available: https://www.eurelectric.org/in-detail/cybersecurity-in-the-power-sector/.
Eurelectric, A Snapshot of Cybersecurity in the EU, Nov. 2024. [Online]. Available: https://www.eurelectric.org/wp-content/uploads/2024/11/A-Eurelectric-snapshot-of-Cybersecurity-2024-11-18-FINAL.pdf.
X. Ou, W.F. Boyer, M.A. McQueen, “A scalable approach to attack graph generation,” Proc. of the 13th ACM Conf. on Computer and Communications Security, pp. 336–345, 2006.
R. Ahmari, V. Hemmati, A. Mohammadi, P. Kebria, M. N. Mahmoud, and A. Homaifar, “A data-driven approach for UAV-UGV integration,” Automation, Robotics & Communications for Industry 4.0/5.0, vol. 77, 2025.
R. Ahmari, V. Hemmati, A. Mohammadi, M. Mynuddin, P. Kebria, M. N. Mahmoud, and A. Homaifar, “Evaluating Trojan attack vulnerabilities in autonomous landing systems for urban air mobility,” Automation, Robotics & Communications for Industry 4.0/5.0, vol. 80, 2025.
B. Wang, X. Kestelyn, E. Kharazian, and A. Grolet, “Application of normal form theory to power systems: A proof of concept of a novel structure-preserving approach,” Proc. IEEE Power & Energy Society General Meeting (PESGM), pp. 1–5, July 2024.
F. S. Banitaba, S. Aygun, and M. H. Najafi, “Late breaking results: Fortifying neural networks: Safeguarding against adversarial attacks with stochastic computing,” arXiv preprint, arXiv:2407.04861, 2024.
K. Sheida, M. Seyedi, M. A. Afridi, F. Ferdowsi, and M. J. Khattak, “Resilient control for islanded hybrid DC microgrid integrating piezoelectric, solar and battery,” in 2024 56th North American Power Symposium (NAPS), Oct. 2024, pp. 1–6.
M. Elsayed, M. Erol-Kantarci, B. Kantarci, L. Wu, and J. Li, “Low-latency communications for community resilience microgrids: A reinforcement learning approach,” IEEE Trans. Smart Grid, vol. 11, no. 2, pp. 1091–1099, 2020.
M. Zhang, Z. Wu, J. Yan, R. Lu, and X. Guan, “Attack-resilient optimal PMU placement via reinforcement learning guided tree search in smart grids,” IEEE Trans. Inf. Forensics Security, vol. 17, pp. 1919–1929, 2022.
F. Wei, Z. Wan, and H. He, “Cyber-attack recovery strategy for smart grid based on deep reinforcement learning,” IEEE Trans. Smart Grid, vol. 11, no. 3, pp. 2476–2486, 2020.
H. Zhang et al., “Resilient optimal defensive strategy of TSK fuzzy-model-based microgrids’ system via a novel reinforcement learning approach,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 4, pp. 1921–1931, 2023.
H. Zhang, D. Yue, C. Dou, and G. P. Hancke, “Resilient optimal defensive strategy of micro-grids system via distributed deep reinforcement learning approach against FDI attack,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 1, pp. 598–608, 2024.
M. Etezadifar, H. Karimi, A. G. Aghdam, and J. Mahseredjian, “Resilient event detection algorithm for non-intrusive load monitoring under non-ideal conditions using reinforcement learning,” IEEE Trans. Ind. Appl., vol. 60, no. 2, pp. 2085–2094, 2024.
A. Sahu, V. Venkatraman, and R. Macwan, “Reinforcement learning environment for cyber-resilient power distribution system,” IEEE Access, vol. 11, pp. 127216–127228, 2023.
L. Zeng, D. Qiu, and M. Sun, “Resilience enhancement of multi-agent reinforcement learning-based demand response against adversarial attacks,” Appl. Energy, vol. 324, p. 119688, 2022.
N. E. Fard and R. R. Selmic, “Data transmission resilience to cyber-attacks on heterogeneous multi-agent deep reinforcement learning systems,” in Proc. 17th Int. Conf. Control, Autom., Robot. Vis. (ICARCV), 2022, pp. 758–764.
P. Chen, S. Liu, B. Chen, and L. Yu, “Multi-agent reinforcement learning for decentralized resilient secondary control of energy storage systems against DoS attacks,” IEEE Trans. Smart Grid, vol. 13, no. 3, pp. 1739–1750, 2022.
Y. Guo, L. Wang, Z. Liu, and Y. Shen, “Reinforcement-learning-based dynamic defense strategy of multistage game against dynamic load altering attack,” Int. J. Electr. Power Energy Syst., vol. 131, p. 107113, 2021.
Y. Huang, L. Huang, and Q. Zhu, “Reinforcement learning for feedback-enabled cyber resilience,” Annu. Rev. Control, vol. 53, pp. 273–295, 2022.
R. Kumar and M. Stoelinga, “Quantitative security and safety analysis with attack-fault trees,” in Proc. 18th IEEE Int. Symp. High Assurance Syst. Eng. (HASE), 2017, pp. 25–32.
X. Liu et al., “Cyber security risk management for connected railroads,” Tech. Rep., U.S. Dept. Transp., Fed. Railroad Admin., 2020.
R. Kumar, B. Narra, R. Kela, and S. Singh, “AFMT: Maintaining the safety-security of industrial control systems,” Comput. Ind., vol. 136, p. 103584, 2022.
É. André, D. Lime, M. Ramparison, and M. Stoelinga, “Parametric analyses of attack-fault trees,” Fundam. Inform., vol. 182, no. 1, pp. 69–94, 2021.
R. Groner et al., “Model-based generation of attack-fault trees,” in Proc. Int. Conf. Comput. Safety, Reliab. Secur., 2023, pp. 107–120.
S. M. Nicoletti, M. Lopuhaä-Zwakenberg, E. M. Hahn, and M. Stoelinga, “Querying fault and attack trees: Property specification on a water network,” in Proc. Annu. Rel. Maintain. Symp. (RAMS), 2024, pp. 1–6.
M. Lopuhaä-Zwakenberg, “Quantitative analysis of attack-fault trees via Markov decision processes,” arXiv preprint arXiv:2408.06914, 2024.
A. Volkanovski, M. Èepin, and B. Mavko, “Application of the fault tree analysis for assessment of power system reliability,” Reliab. Eng. Syst. Saf., vol. 94, no. 6, pp. 1116–1127, 2009.
G. Song, H. Chen, and B. Guo, “A layered fault tree model for reliability evaluation of smart grids,” Energies, vol. 7, no. 8, pp. 4835–4857, 2014.
R. Meyur, “A Bayesian attack tree based approach to assess cyber-physical security of power system,” in Proc. IEEE Texas Power Energy Conf. (TPEC), 2020, pp. 1–6.
Z. Tian et al., “A security model of SCADA system based on attack tree,” in Proc. IEEE 3rd Conf. Energy Internet Energy Syst. Integr. (EI2), 2019, pp. 2653–2658.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright (c) 2025 Journal of Cyber Security and Mobility