IaaS Cloud Adaptive Anomaly Detection Based on the DQN Algorithm
DOI:
https://doi.org/10.13052/jwe1540-9589.2543Keywords:
Deep Q-network, Abnormal detection, Convolutional neural network, Time Convolutional Network, Cloud computingAbstract
Aiming at the challenges of anomaly detection of virtual machine memory, network, CPU and hard disk in the IaaS cloud environment, this study proposes an adaptive anomaly detection system based on a deep Q-network. The system constructs a hierarchical detection framework: a spatio-temporal feature extraction module via fused temporal convolutional networks (TCN) for sequential pattern mining and convolutional neural networks (CNN) for cross-metric correlation learning; a transfer learning module to enhance generalization; and a deep Q-network (DQN) based central controller that dynamically adjusts detection parameters through reinforcement learning. This architecture integrates with cloud workload schedulers by operating at the VM-level (anomaly detection) and edge-server level (DQN control), minimizing core network overhead. Experiments show that the research method achieves a detection accuracy rate of 99.8% in the benchmark test, with an F1 score of 98.7%, which is significantly superior to the accuracy rate of 96.5% of the single convolutional neural network, 92.3% of the multi-layer perceptron, and 97.8% of Google Net. The transfer training experiments show that the accuracy rate of the untuned model on the new dataset is only 70% to 80%, while the detection accuracy can be stably improved to 98% through the adaptive system driven by the DQN. The system shows low volatility during the dynamic adjustment process. The number of training iterations is reduced by 32.3% to 69.8% compared with the traditional static model, indicating that the research method does not affect the time complexity. Research shows that this framework effectively solves the problem of insufficient adaptability of static models to unknown data in the cloud environment through the collaborative mechanism of spatio-temporal feature extraction and reinforcement learning decision-making, providing intelligent operation and maintenance solutions for fields with high reliability requirements such as finance and healthcare.
Downloads
References
C. Gan, Q. Feng, X. Zhang, Z. Zhang, and Q. Zhu, “Dynamical propagation model of malware for cloud computing security,” IEEE Access, vol. 8, no. 1, pp. 20325-20333, Jan. 2020.
H. Xu, G. Pang, Y. Wang, and Y. Wang, “Deep isolation forest for anomaly detection,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 12, pp. 12591–12604, Apr. 2023.
R. Ranjbarzadeh, N. Tataei Sarshar, S. Jafarzadeh Ghoushchi, M. Saleh Esfahani, M. Parhizkar, Y. Pourasad, et al., “MRFE-CNN: Multi-route feature extraction model for breast tumor segmentation in Mammograms using a convolutional neural network,” Ann. Oper. Res., vol. 328, no. 1, pp. 1021–1042, May 2023.
Y. Baghoussi, C. Soares, and J. Mendes-Moreira, “Corrector LSTM: Built-in training data correction for improved time-series forecasting,” Neural Comput. Appl., vol. 36, no. 26, pp. 16213–16231, May 2024.
Z. Li, L. Tian, Q. Jiang, and X. Yan, “Fault diagnostic method based on deep learning and multimodel feature fusion for complex industrial processes,” Ind. Eng. Chem. Res., vol. 59, no. 40, pp. 18061–18069, Oct. 2020.
J. Liu, J. Bai, H. Li, and B. Sun, “Improved LSTM-based abnormal stream data detection and correction system for internet of things,” IEEE Trans. Ind. Inform., vol. 18, no. 2, pp. 1282–1290, Feb. 2022.
J. Hu, X. Zhang, and S. Maybank, “Abnormal driving detection with normalized driving behavior data: A deep learning approach,” IEEE Trans. Veh. Technol., vol. 69, no. 7, pp. 6943–6951, May 2020.
M. Raichura, N. Chothani, and D. Patel, “Efficient CNN-XGboost technique for classification of power transformer internal faults against various abnormal conditions,” IET Generation, Transm. & Distrib., vol. 15, no. 5, pp. 972–985, Mar. 2021.
Y. Xin, J. Wang, and H. Wei, “Hybrid fuzzy integrated convolutional neural network (HFICNN) for similarity feature recognition problem in abnormal netflow detection,” Neurocomputing, vol. 415, no. 1, pp. 332–346, Jul. 2020.
Y. T. Quek, W. A. Tso, W. L. Woo, N. T. Koh, and L. L. Koh, “Deep Q-network implementation for simulated autonomous vehicle control,” IET Intel. Transp. Syst., vol. 15, no. 7, pp. 875–885, Jul. 2021.
Y. Zheng, Q. Sun, Z. Chen, M. Sun, J. Tao, and H. Sun, “Deep Q-Network based real-time active disturbance rejection controller parameter tuning for multi-area interconnected power systems,” Neurocomputing, vol. 460, no. 10, pp. 360–373, Oct. 2021.
J. Mei, X. Wang, K. Zheng, G. Boudreau, A. B. Sediq, and H. Abou-Zeid, “Intelligent radio access network slicing for service provisioning in 6G: A hierarchical deep reinforcement learning approach,” IEEE Trans. Commun., vol. 69, no. 9, pp. 6063–6078, Jun. 2021.
Z. Ke, Z. Li, Z. J. Cao, and P. Liu, “Enhancing transferability of deep reinforcement learning-based variable speed limit control using transfer learning,” IEEE Trans. Intel. Transp. Syst., vol. 22, no. 7, pp. 4684–4695, May 2020.
L Bommes, Hoffmann M, Claudia Buerhop-Lutz, T Pickel, J Hauch, C Brabec, A Maier, I Marius Peters. “Anomaly detection in IR images of PV modules using supervised contrastive learning,” Prog. Photovoltaics, vol. 30, no. 6, pp. 597–614, 2022.
S F Kate, H C Marc, Nesar R, L Francois, L Alexie, L Yifei, H Song, P J Xavier, “Anomaly detection in Hyper Suprime-Cam galaxy images with generative adversarial networks,” Mon. Not. R. Astron. Soc., vol. 508, no. 2, pp. 2946–2963, 2021.
B. Mohammed, I. Awan, H. Ugail, and M. Younas, “Failure prediction using machine learning in a virtualised HPC system and application,” Cluster Comput., vol. 22, no. 2, pp. 471–485, Jun. 2019.
F. Cerveira, R. Barbosa, H. Madeira, and F. Araujo, “The effects of soft errors and mitigation strategies for virtualization servers,” IEEE Trans. Cloud Comput., vol. 10, no. 2, pp. 1065–1081, Feb. 2020.
Z. Zhang, J. Wen, J. Zhang, X. Cai, and L. Xie, “A many objective-based feature selection model for anomaly detection in cloud environment,” IEEE Access, vol. 8, no. 3, pp. 60218–60231, Mar. 2020.
S. D. Hallgrímsson, H. H. Niemann, and M. Lind, “Unsupervised isolation of abnormal process variables using sparse autoencoders,” J. Process Control, vol. 99, no. 9, pp. 107–119, Mar. 2021.
M. Razian, M. Fathian, H. Wu, A. Akbari, and R. Buyya, “SAIoT: Scalable anomaly-aware services composition in cloudiot environments,” IEEE Internet Things J., Mar. 2021.
M. Fahim andA. Sillitti, “Anomaly detection, analysis and prediction techniques in IoT environment: A systematic literature review,” IEEE Access, vol. 7, no. 1, pp. 81664–81681, Jan. 2019.
X. Chen, J. Chen, D. Zhao, and X. Jin, “Anomaly detection based on IO sequences in a virtual machine with the markov mode,” J. Tsinghua Univ., vol. 58, no. 4, pp. 395–401, Apr. 2018.
E. Ataie, R. Entezari-Maleki, S. E. Etesami, B. Egger, D. Ardagna, and A. Movaghar, “Power-aware performance analysis of self-adaptive resource management in IaaS clouds,” Future Gener. Comput. Syst., vol. 86, no. 11, pp. 134–144, Mar. 2018.
Y. Hui, “A virtual machine anomaly detection system for cloud computing infrastructure,” J. Supercomput., vol. 74, no. 11, pp. 6126–6134, Nov. 2018.
Z. Chen, “Research on internet security situation awareness prediction technology based on improved RBF neural network algorithm,” J. Comput. Cogn. Eng., vol. 1, no. 3, pp. 103–108, Mar. 2022.
A. Vms and B. Kse, “An improved dynamic fault tolerant management algorithm during VM migration in cloud data center,” Future Gener. Comput. Syst., vol. 98, no. 9, pp. 35–43, Sep. 2019.
A. Zhu, Z. Tang, Z. Wang, Y. Zhou, S. Chen, F. Hu, and Y. Li, “Wi-ATCN: Attentional temporal convolutional network for human action prediction using wifi channel state information,” IEEE J. Sel. Top. Signal Process., vol. 16, no. 4, pp. 804–816, Jun. 2022.
Z. Shen, Y. Zhang, J. Lu, J. Xu, and G. Xiao, “A novel time series forecasting model with deep learning,” Neurocomputing, vol. 396, no. 10, pp. 301–313, Apr. 2019.
J. An, G. Liang, L. Wei, Z. Fu, R. Ping, X. Liu, and L. Tao, “IGAGCN: Information geometry and attention-based spatiotemporal graph convolutional networks for traffic flow prediction,” Neural Networks, vol. 143, no. 722, pp. 355–367, Jun. 2021.
Y. Chen, Y. Kang, Y. Chen, and Z. Wang, “Probabilistic forecasting with temporal convolutional neural network,” Neurocomputing, vol. 399, no. 1, pp. 491–501, Mar. 2020.
Z. Yan, J. Ge, Y. Wu, L. Li, and T. Li, “Automatic virtual network embedding: A deep reinforcement learning approach with graph convolutional networks,” IEEE J. Sel. Are. Commun., vol. 38, no. 6, pp. 1040–1057, Apr. 2020.
T. Zhang, K. Zhu, and J. Wang, “Energy-efficient mode selection and resource allocation for D2D-enabled heterogeneous networks: A deep reinforcement learning approach,” IEEE Trans. Wirel. Commun., vol. 20, no. 2, pp. 1175–1187, Oct. 2020.
S. Liu, G. Tian, Y. Zhang, M. Zhang, and S. Liu, “Active object detection based on a novel deep Q-learning network and long-term learning strategy for service robot,” IEEE Trans. Ind. Electron., vol. 69, no. 6, pp. 5984–5993, Jun. 2021.
