Detecting APT Attacks Based on Network Traffic Using Machine Learning

Keywords: Advanced Persistent Threat, APT attack detection, Network traffic, domain, abnormal behaviour, machine learning


Advanced Persistent Threat (APT) attacks are a form of malicious, intentionally and clearly targeted attack. By using many sophisticated and complicated methods and technologies to attack targets in order to obtain confidential and sensitive information. In fact, in order to detect APT attacks, detection systems often need to apply many parallel and series techniques in order to make the most of the advantages as well as minimize the disadvantages of each technique. Therefore, in this paper, we propose a method of detecting APT attacks based on abnormal behaviors of Network traffic using machine learning. Accordingly, in our research, the abnormal behavior of APT attacks in Network Traffic will be defined on both components: Domain and IP. Then, these behaviors are evaluated and classified based on the Random Forest classification algorithm to conclude about the behavior of APT attacks. Details of the definition of abnormal behaviors of the Domain and IP will be presented in section 3.2 of the paper.  The synchronous APT attack detection method proposed in this paper is a novel approach, which will help information security systems detect quickly and accurately signs of the APT attack campaign in the organization. The experimental results presented in section 4 will demonstrate the effectiveness of our proposed method.


Download data is not yet available.

Author Biography

Cho Do Xuan, Information Assurance dept. FPT University, Hanoi, Vietnam

Cho Do Xuan is currently a lecturer at the Faculty of Information Technology at Posts and Telecommunications Institute of Technology and FPTU in Vietnam In 2008, received a bachelor’s degree in the Saint Petersburg Electrotechnical University “LETI” on a specialty “Computer science and computer facilities”, Russia. In 2010, graduated a masters from the Saint Petersburg Electrotechnical University “LETI” on a specialty “Computer science and computer facilities”, Russia. In 2013, received a PhD in the Saint Petersburg Electrotechnical University “LETI”, on a specialty CAD. Russia. Area of scientific interests – modeling, control systems, algorithmization, information security. E-mail: and


Quintero Bonilla, Santiago & Rey, Ángel. A New Proposal on the Advanced Persistent Threat: A Survey. Applied Sciences. 2020, 10(11), pp. 38-74.

Adel, A., Ankur, C., Sowmya, M., Dijiang Huang, H.: A Survey on Advanced Persistent Threats: Techniques, Solutions, Challenges, and Research Opportunities. IEEE Commu. Sur. & Tu. PP99(1-1), 1–29 (2019).

Zimba, Aaron ; Chen, Hongsong ; Wang, Zhaoshun ; Chishimba, Mumbi. Modeling and detection of the multi-stages of Advanced Persistent Threats attacks based on semi-supervised learning and complex networks characteristics. Future Generation Computer Systems. Volume 106, 2020, pp. 501-517.

Sadegh, M.M., Rigel, Gj., Birhanu, E., Ramachandran, S.: HOLMES: Real-time APT Detection through Correlation of Suspicious Information Flows. In: 2019 IEEE Symposium on Security and Privacy, pp. 1137-1152, San Francisco, CA, USA, 19-23 May 2019.

Lajevardi, Amir ; Amini, Morteza. A semantic-based correlation approach for detecting hybrid and low-level APTs. Future Generation Computer Systems. Vol 96, 2019, pp. 64-88.

Weina, N., Xiaosong, Z., GuoWu, Y., Jianan, Z., Zhongwei, R.: Identifying APT Malware Domain Based on Mobile DNS Logging. Mat. Pro. in. Eng. 2, 1- 9 (2017).

Zhao, G., Xu, K., Xu, L., Wu, B.: Detecting APT malware infections based on malicious DNS and traffic analysis. IEEE Access. 3, 1132–1142 (2015).

Do Xuan Cho, Ha Hai Nam. A Method of Monitoring and Detecting APT Attacks Based on Unknown Domains. Pro. Com. Sci. 150, 316-323 (2019).

Jiazhong Lu, Kai Chen, Zhongliu Zhuo, XiaoSong Zhang. A temporal correlation and traffic analysis approach for APT attacks detection. Cluster Computing (2017). pp 1–12.

Guanghua Yan, Qiang Li , Dong Guo, Xiangyu Meng. Discovering Suspicious APT Behaviors by Analyzing DNS Activities. Sensors 2020, 20, 731; doi:10.3390/s20030731.

R. Vinayakumara, K.P. Somana, P. Poornachandranb. Detecting malicious domain names using deep learning approaches at scale. Journal of Intelligent and Fuzzy Systems. 2018, 34, 1355-1367.

Van Can, Nguyen et al. A New Method to Classify Malicious Domain Name Using Neutrosophic Sets in DGA Botnet Detection. Journal of Intelligent and Fuzzy Systems. 2020, 36 4223 – 4236.

Cho Do Xuan, Hoa Dinh Nguyen, Hoang Mai Dao. APT attack detection based on flow network analysis techniques using deep learning. Journal of Intelligent & Fuzzy Systems, vol. Pre-press, no. Pre-press, pp. 1-17, 2020.

Cho Do Xuan, Lai Van Duong and Tisenko Victor Nikolaevich, “Detecting C&C Server in the APT Attack based on Network Traffic using Machine Learning” International Journal of Advanced Computer Science and Applications(IJACSA), 11(5), 2020.

Shai, S.S., Shai B.D.: Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press (2014).

Leo, B.: Random Forests. Ma. lear. 45 (1), 5- 32 (2001).

Xuan, Cho. Malicious domain detection based on DNS query using Machine Learning. International Journal of Emerging Trends in Engineering Research. No 8, 2020, pp. 1809-1814.7

OpenDNS public domain lists of domain names for training/testing classifier. [access date 1/4/3018]

Malware Domain List. [access date 1/4/2020].

Join the fight against phishing. [access date 1/4/2020]

Alexa - Top Sites for Countries. [access date 1/4/2020]

Public-domain-lists. [access date 3/4/2020].

APTNotes - Github Repo. [access date 3/4/2020].

APTNotes - Website Targeted. [access date 3/4/2020].

Cyber Attacks Logbook (Kaspersky) [access date 3/4/2020].

DeepEnd Research: List of malware pcaps, samples, and indicators for the Library of Malware Traffic Patterns. [access date 3/4/2020].

Data Science and Artificial Intelligence: Architecture, Use Cases, and Challenge