Robust Deep Learning Empowered Real Time Object Detection for Unmanned Aerial Vehicles based Surveillance Applications


  • C. Prasanna Ranjith Faculty in Information Technology Department, University of Technology and Applied Sciences, Shinas, Sultanate of Oman
  • Bhalchandra M. Hardas Electronics Engineirng Dept, Shri Ramdeobaba College of Enginering and Management, Nagpur, India
  • M. Syed Khaja Mohideen Department of Information Technology, University of Technology and Applied Science – Salalah, Sultanate of Oman
  • N. Nijil Raj Department of Computer Science and Engineering Younus College of Engineering and Technology, Kollam, India
  • Nismon Rio Robert Department of Computer Science, Christ University, Bangalore, India
  • Prakash Mohan School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India



Surveillance, Deep Learning, Unmanned Aerial Vehicles, Object Detection, Computer vision, Image Processing


Surveillance is a major stream of research in the field of Unmanned Aerial Vehicles (UAV), which focuses on the observation of a person, group of people, buildings, infrastructure, etc. With the integration of real time images and video processing approaches such as machine learning, deep learning, and computer vision, the UAV possesses several advantages such as enhanced safety, cheap, rapid response, and effective coverage facility. In this aspect, this study designs robust deep learning based real time object detection (RDL-RTOD) technique for UAV surveillance applications. The proposed RDL-RTOD technique encompasses a two-stage process namely object detection and objects classification. For detecting objects, YOLO-v2 with ResNet-152 technique is used and generates a bounding box for every object. In addition, the classification of detected objects takes place using optimal kernel extreme learning machine (OKELM). In addition, fruit fly optimization (FFO) algorithm is applied for tuning the weight parameter of the KELM model and thereby boosts the classification performance. A series of simulations were carried out on the benchmark dataset and the results are examined under various aspects. The experimental results highlighted the supremacy of the RDL-RTOD technique over the recent approaches in terms of several performance measures.


Download data is not yet available.

Author Biographies

C. Prasanna Ranjith, Faculty in Information Technology Department, University of Technology and Applied Sciences, Shinas, Sultanate of Oman

C. Prasanna Ranjith Obtained Ph.D. in Computer Science from Bharathidasan University, Trichy, India in Dec 2017. Received Bachelor, Master of Science and Master of Philosophy in computer science in 1995,1997 & 2004 from Bharathidasan University, Trichy, India.Have more than 23 years of teaching experience at different Colleges and Universities in India, Libya, and Sultanate of Oman. Presently a Faculty & Research Chair, department of Information Technology at University of Technology and Applied Sciences, Shinas, Oman since October 2011. Organized various Workshops, Seminars and symposiums towards professional Research and development. Published many research papers in international journals and conferences of high repute. Hold ample experience in the field of Web Designing having widespread familiarity in ASP.Net (C#) & MS SQL Server. Awarded BEST FACULTY for 3 consecutive years. My research interests include Machine Learning, Deep Learning, Nature Inspired Algorithms, Soft Computing, Parallel Algorithms, Genetic Algorithms and Ad Hoc Networks.

Bhalchandra M. Hardas, Electronics Engineirng Dept, Shri Ramdeobaba College of Enginering and Management, Nagpur, India

Bhalchandra M. Hardas has done his M.Tech and Ph D in Electronics Engineering from Rashtrsant Tukadoji Maharaj Nagpur University, Nagpur. His research contribution includes development of statistical maximum value distribution algorithm for AWGN, Rayleigh and Rician channel in MIMO-OFDM. His current interests are in FPGA implementation of novel PAPR reduction algorithm. He is member of ISTE and IETE. He is recipient of Best teacher award of RTM Nagpur University in year 2017–2018. Presently he is working as Assistant Professor in Electronics Engineering department. He is coordinator of IPR cell of Shri Ramdeobaba college of Engineering and Management, Nagpur.

M. Syed Khaja Mohideen, Department of Information Technology, University of Technology and Applied Science – Salalah, Sultanate of Oman

M. Syed Khaja Mohideen Completed Undergraduate from MS University in 1997, Masters degree in Computer Science in 2000, MPhil(Computer Science) in 2004 and the PhD degree in Computer Science in 2019 from Bharatidasan University Trichy. Worked as Assistant Professor in Computer Science at Bishop Heber College till 2007 and Omar Mukthar University, Libya 2007 to 2011. Presently working in University of Technology and Applied Sciences – Salalah, Sultanate of Oman. Current research interest includes ad-hoc networks.

N. Nijil Raj, Department of Computer Science and Engineering Younus College of Engineering and Technology, Kollam, India

N. Nijil Raj, currently working as professor and head, department of CSE, Younus College of engineering and technology, Kollam, affiliated to APJ technological university, Thiruvananthapuram, Kerala, He has published more than 15 papers in national and international journals. He was post graduated from M. S. University, Tamilnadu in M.Tech, MCA and MBA from MG University Kottayam, Kerala. His area interest is bioinformatics, Machine Learning and AI, and Image Processing. He is having 18 years of teaching experience in UG and PG level. He has completed the Doctoral Degree from M.S. University, Tamilnadu, in Computer Science and Information Technology.

Nismon Rio Robert, Department of Computer Science, Christ University, Bangalore, India

Nismon Rio Robert is currently working as an Assistant Professor of Computer Science, CHRIST (Deemed to be University), Bangalore, India, since 2019. He holds a Ph.D. Degree in Computer Science from Bishop Heber College affiliated to Bharathidasan University, Tiruchirappalli, India in 2019. He received financial support from University Grants Commission (UGC) of the research work. His research interest is route optimization in wireless networks, Mobile Ad-hoc Networks, Internet of Things and other related topics. He has presented 30 papers in National and International conferences/journals. Moreover, he has published 6 Indian Patents in the field of communication technologies. He has delivered technical lectures in various institutions.

Prakash Mohan, School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India

Prakash Mohan, SM IEEE, Professor is working in the Department of Computer Science and Engineering Karpagam College of Engineering, Coimbatore, Tamil Nadu, India. He received his Doctor of Philosophy in 2014 from Jawaharlal Nehru Technological University Hyderabad. He has more than 20+ years of experience in teaching and research. His area of interest includes Data Analytics, Big Data, and Machine Learning. He has published more than 50 research papers in International Journals/ Conferences. He has served as a lead guest editor for the journals Inderscience, EAI Endorsed Transition, Bentham publisher, and Tech science press. He is an Editorial Review Board of IGI Global. He has reviewed journals from the publishers IEEE, Elsevier, Springer, Taylor & Francis, and Inderscience. He has received awards from the Computer Society of India towards the Highest hosting CSI Events and is an Active CSI member in Coimbatore Chapter. He received grants from CSIR and CSI to conduct Conference, Seminar and Students State Level Convention. He is a member of IEEE, Association for Computing Machinery, Computer Society of India, ISTE, and IAENG.


Bhaskaranand,M, a. J. (n.d.). Low-complexity video encoding for UAV reconnaissance and surveillance. Proc. IEEE Military Communications Conference (MILCOM), pp. 1633-1638, 2011.

Lim, a. S. Monocular Localization of a moving person onboard a Quadrotor MAV. Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 2182–2189, 2015.

I. Sa, S. H. Outdoor flight testing of a pole inspection UAV incorporating highspeed vision. Springer Tracts in Advanced Robotics, pp. 107–121, 2015.

Christos Kyrkou, G. P. DroNet: Efficient convolutional neural network detector for realtime UAV applications. Design, Automation and Test in Europe Conference and Exhibition (DATE), 2018.

J. Gokul Anand, “Trust based optimal routing in MANET’s,” 2011 International Conference on Emerging Trends in Electrical and Computer Technology, Nagercoil, India, 2011, pp. 1150–1156, doi: 10.1109/ICETECT.2011.5760293.

S.Divyabharathi,“Large scale optimization to minimize network traffic using MapReduce in big data applications”. International Conference on Computation of Power, Energy Information and Communication (ICCPEIC), pp. 193–199, April 2016. DOI: 10.1109/ICCPEIC.2016.7557196.

Paulraj, D 2020, ‘An Automated Exploring And Learning Model For Data Prediction Using Balanced CA-SVM’, Journal of Ambient Intelligence and Humanized Computing, Vol. 12, no. 5, April 2020, DOI:

Berlin, M.A., Tripathi, S. et al. IoT-based traffic prediction and traffic signal control system for smart city. Soft Computing (2021).

P. Subbulakshmi, “Mitigating eavesdropping by using fuzzy based MDPOP-Q learning approach and multilevel Stackelberg game theoretic approach in wireless CRN”, Cognitive Systems Research, Vol. 52, pp. 853–861, 2018. doi: 10.1016/j.cogsys.2018.09.021.

V. D. Ambeth Kumar, S. Malathi, Abhishek Kumar, Prakash M and Kalyana C. Veluvolu, “Active Volume Control in Smart Phones Based on User Activity and Ambient Noise,” Sensors 2020, 20(15), 4117. doi: 10.3390/s20154117.

Sambit Satpathy, Sanchali Das, Swapan Debbarma, “A new healthcare diagnosis system using an IoT-based fuzzy classifier with FPGA”, Journal of Supercomputing, vol. 76, no. 8, pp. 5849–5861, 2020. doi: 10.1007/s11227-019-03013-2.

Jangwon, L., Wang, J., Crandall, D., Šabanovic, S. and Fox, G., 2017. Real-time object detection for unmanned aerial vehicles based on cloud-based convolutional neural networks. Journal Concurrency and Computation: Practice and Experience, 29(6).

F. S. Leira, T. A. Johansen, and T. I. Fossen, “Automatic detection, classification and tracking of objects in the ocean surface from UAVs using a thermal camera,” in Proc. IEEE Aerospace Conference, pp. 1–10, 2015.

J. Engel, J. Sturm, and D. Cremers, “Scale-aware navigation of a low-cost quadrocopter with a monocular camera,” Robotics and Autonomous Systems, vol. 62, no. 11, pp. 1646–1656, 2014.

D. Ciresan, A. Giusti, L. M. Gambardella, and J. Schmidhuber, “Deep neural networks segment neuronal membranes in electron microscopy images,” in Advances in neural information processing systems, pp. 2843–2851, 2012.

Alam, M.S., Natesha, B.V., Ashwin, T.S. and Guddeti, R.M.R., 2019. UAV based cost-effective real-time abnormal event detection using edge computing. Multimedia tools and Applications, 78(24), pp. 35119–35134.

Vaddi, S., 2019. Efficient object detection model for real-time UAV applications (Doctoral dissertation, Iowa State University).

Zhang, H., Liptrott, M., Bessis, N. and Cheng, J., 2019, September. Real-time traffic analysis using deep learning techniques and uav based video. In 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) (pp. 1–5). IEEE.

Nguyen, A.Q., Nguyen, H.T., Tran, V.C., Pham, H.X. and Pestana, J., 2021, January. A Visual Real-time Fire Detection using Single Shot MultiBox Detector for UAV-based Fire Surveillance. In 2020 IEEE Eighth International Conference on Communications and Electronics (ICCE) (pp. 338–343). IEEE.

Lai, Y.C. and Huang, Z.Y., 2020. Detection of a Moving UAV Based on Deep Learning-Based Distance Estimation. Remote Sensing, 12(18), p. 3035.

Dong, J., Ota, K. and Dong, M., 2021. UAV-based Real-Time Survivor Detection System in Post-disaster Search and Rescue Operations. IEEE Journal on Miniaturization for Air and Space Systems.

Redmon, J., and Farhadi, A. (2017). YOLO9000: Better, Faster, Stronger. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6517–6525.

Loey, M., Manogaran, G., Taha, M.H.N. and Khalifa, N.E.M., 2021. Fighting against COVID-19: A novel deep learning model based on YOLO-v2 with ResNet-152 for medical face mask detection. Sustainable cities and society, 65, p. 102600.

Lu, J., Huang, J. and Lu, F., 2019. Distributed kernel extreme learning machines for aircraft engine failure diagnostics. Applied Sciences, 9(8), p. 1707.

Ding, G., Dong, F. and Zou, H., 2019. Fruit fly optimization algorithm based on a hybrid adaptive-cooperative learning and its application in multilevel image thresholding. Applied Soft Computing, 84, p. 105704.

Sun, C., Zhan, W., She, J. and Zhang, Y., 2020. Object detection from the video taken by drone via convolutional neural networks. Mathematical Problems in Engineering, 2020.





New Trends in Real-Time Image and Video Processing for Surveillance and Security