ISSN: 2245-4578 (Online Version) ISSN:2245-1439 (Print Version)
AI-Based Malware Detection and Classification Algorithms
PDF
HTML

Keywords

Graph Neural Network
Malicious Software Detection
Attention Mechanism
Control Flow Diagram
Adversarial Training

How to Cite

[1]
Z. . Zhu, “AI-Based Malware Detection and Classification Algorithms”, JCSANDM, vol. 15, no. 03, pp. 525–548, Jun. 2026.

Abstract

Malware exhibits characteristics such as rapid variant evolution, sophisticated obfuscation techniques, and frequent zero-day attacks. Existing detection methods suffer from issues like insufficient feature extraction, weak generalization capabilities, and difficulty in capturing code semantic information. This paper proposes a malware detection and classification algorithm based on the fusion of Graph Neural Networks (GNN) and attention mechanisms. First, this paper transforms the control flow graph and function call graph of malware into a heterogeneous graph structure, extracting node and edge features. Second, it employs a Graph Convolutional Network (GCN) for multi-layer feature aggregation, introducing a multi-head attention mechanism to adaptively learn the weights of key code snippets. Then, it reduces dimensionality and integrates global features through a graph pooling layer, utilizing a fully connected layer for binary classification detection and multi-class family identification of malware. Finally, adversarial training is applied to enhance the model’s robustness. Verified on a public dataset containing 15000 samples, the overall detection accuracy reached 98.7%, the recall rate reached 98.8%, and the detection rate for confused samples increased to 96.1%. The experimental results show that this method can effectively identify variants of malicious software and has strong practical value.

https://doi.org/10.13052/jcsm2245-1439.1531
PDF
HTML

References

Shi Zhibin, Sun Wenqi, Dou Jianmin, and Yu Mengyang, “Research on malicious software detection based on word embedding and feature fusion,” Information Security Research, vol. 11, no. 5, pp. 412–419, 2025.

Xiong Zhi, Liu Fang, and Wang Yixuan, “Android malware detection based on feature weighting for classifiers,” Computer Engineering and Science, vol. 47, no. 9, pp. 1598–1608, 2025.

J. Kim, Y. Ban, E. Ko, et al., “MAPAS: A practical deep learning-based Android malware detection system,” International Journal of Information Security, vol. 21, no. 4, pp. 725–738, 2022.

Xie Lixia, Wei Chenyang, Yang Hongyu, Hu Ze, and Cheng Xiang, “Malicious software detection method based on multi-dimensional dynamic weighted alpha image fusion and feature enhancement,” Acta Sinica, vol. 53, no. 3, pp. 849–863, 2025.

Yu Mengyang, Shi Zhibin, Hao Weize, and Zhang Shujuan, “Malicious software detection based on VAE and API behavior feature extraction,” Computer Engineering and Design, vol. 46, no. 2, pp. 464–471, 2025.

Wang Shengjie, Zhang Qinghong, and Wang Ziwei, “Interpretability of intelligent fusion models in malicious software detection,” Science, Technology, and Engineering, vol. 25, no. 23, pp. 9892–9899, 2025.

U. H. Tayyab, F. B. Khan, M. H. Durad, et al., “A survey of the recent trends in deep learning-based malware detection,” Journal of Cybersecurity and Privacy, vol. 2, no. 4, pp. 800–829, 2022.

M. G. Gaber, M. Ahmed, and H. Janicke, “Malware detection with artificial intelligence: A systematic literature review,” ACM Computing Surveys, vol. 56, no. 6, pp. 1–33, 2024.

B. R. Maddireddy and B. R. Maddireddy, “Automating malware detection: A study on the efficacy of AI-driven solutions,” Journal of Environmental Sciences and Technology, vol. 2, no. 2, pp. 111–124, 2023.

F. Deldar and M. Abadi, “Deep learning for zero-day malware detection and classification: A survey,” ACM Computing Surveys, vol. 56, no. 2, pp. 1–37, 2023.

D. P. Deevi, “Real-time malware detection via adaptive gradient support vector regression combined with LSTM and hidden Markov models,” Journal of Science and Technology, vol. 5, no. 4, 2020.

T. Bilot, N. El Madhoun, K. Al Agha, et al., “A survey on malware detection with graph representation learning,” ACM Computing Surveys, vol. 56, no. 11, pp. 1–36, 2024.

T. Li, Y. Liu, Q. Liu, et al., “A malware propagation prediction model based on representation learning and graph convolutional networks,” Digital Communications and Networks, vol. 9, no. 5, pp. 1090–1100, 2023.

L. Feng, Y. Zhao, W. Zhao, et al., “A comparative review of graph convolutional networks for human skeleton-based action recognition,” Artificial Intelligence Review, vol. 55, no. 5, pp. 4275–4305, 2022.

S. Jia, S. Jiang, S. Zhang, et al., “Graph-in-graph convolutional network for hyperspectral image classification,” IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 1, pp. 1157–1171, 2022.

C. Huang, M. Li, F. Cao, et al., “Are graph convolutional networks with random weights feasible?” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 3, pp. 2751–2768, 2022.

D. Qiu and B. Yang, “Text summarization based on multi-head self-attention mechanism and pointer network,” Complex & Intelligent Systems, vol. 8, no. 1, pp. 555–567, 2022.

Y. Zang, Z. Yu, K. Xu, et al., “Multi-span long-haul fiber transmission model based on cascaded neural networks with multi-head attention mechanism,” Journal of Lightwave Technology, vol. 40, no. 19, pp. 6347–6358, 2022.

S. Li, Y. Xu, W. Jiang, et al., “A modular fault diagnosis method for rolling bearing based on mask kernel and multi-head self-attention mechanism,” Transactions of the Institute of Measurement and Control, vol. 46, no. 5, pp. 899–912, 2024.

D. Wang, Z. Zhang, Y. Jiang, et al., “DM3Loc: Multi-label mRNA subcellular localization prediction and analysis based on multi-head self-attention mechanism,” Nucleic Acids Research, vol. 49, no. 8, e46, 2021.

J. Chen, L. Song, S. Cai, et al., “TLS-MHSA: An efficient detection model for encrypted malicious traffic based on multi-head self-attention mechanism,” ACM Transactions on Privacy and Security, vol. 26, no. 4, pp. 1–21, 2023.

W. Wang, J. Bickford, I. Murynets, R. Subbaraman, A. G. Forte, and G. Singaraju, “Detecting targeted attacks by multilayer deception,” Journal of Cyber Security and Mobility, vol. 2, no. 2, pp. 175–199, 2013, doi: 10.13052/jcsm2245-1439.224.

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2026 Journal of Cyber Security and Mobility

Downloads

Download data is not yet available.