Tiny Drone Tracking Framework Using Multiple Trackers and Kalman-based Predictor





Object Tracking, Unmanned Aerial Vehicles, Drones, Surveillance System


Unmanned aerial vehicles like drones are one of the key development technologies with many beneficial applications. As they have made great progress, security and privacy issues are also growing. Drone tacking with a moving camera is one of the important methods to solve these issues. There are various challenges of drone tracking. First, drones move quickly and are usually tiny. Second, images captured by a moving camera have illumination changes. Moreover, the tracking should be performed in real-time for surveillance applications. For fast and accurate drone tracking, this paper proposes a tracking framework utilizing two trackers, a predictor, and a refinement process. One tracker finds a moving target based on motion flow and the other tracker locates the region of interest (ROI) employing histogram features. The predictor estimates the trajectory of the target by using a Kalman filter. The predictor contributes to keeping track of the target even if the trackers fail. Lastly, the refinement process decides the location of the target taking advantage of ROIs from the trackers and the predictor. In experiments on our dataset containing tiny flying drones, the proposed method achieved an average success rate of 1.134 times higher than conventional tracking methods and it performed at an average run-time of 21.08 frames per second.


Download data is not yet available.

Author Biographies

Sohee Son, Hanbat National University, South Korea

Sohee Son received her B.S. and M.S. degrees from Hanbat National University, Daejeon, Korea, in 2015 and 2017, respectively, from the department of multimedia engineering. Currently, she is working toward Ph.D. degree in the department of multimedia engineering in Hanbat National University. Her research interests include video coding, parallel processing, and computer vision.

Jeongin Kwon, Hanbat National University, South Korea

Jeongin Kwon received a B.S degree in industrial management engineering and a M.S degree in multimedia engineering from Hanbat National University, Daejeon, Korea, in 2014 and 2020, respectively. Currently, she is working on a personal project on deep learning. Her research interests include computer vision, image processing and deep learning.

Hui-Yong Kim, Kyung Hee University, South Korea

Hui-Yong Kim received his B.S., M.S., and Ph.D. degrees from Korea Advanced Institute of Science and Technology (KAIST) in 1994, 1998, and 2004, respectively. He is currently an Associate Professor in the Dept. of Computer Engineering in Kyung Hee University, Yongin, Korea since Mar. 2020. He was also an Associate Professor in the Dept. of Electronics Engineering in Sookmyung Women’s University, Seoul, Korea from Sep. 2019 to Feb. 2020. From 2005 to 2019, prior to joining universities, we worked for Electronics and Telecommunications Research Institute (ETRI), Daejeon, Korea as the Managing Director of Realistic Audio & Video Research Group. From 2003 to 2005, he worked for AddPac Technology Co. Ltd., Seoul as the leader of Multimedia Research Team. He was also an affiliate professor in University of Science and Technology (UST) and was a visiting scholar in the Media Communications Lab. at University of Southern California (USC), USA. His current research interest includes video signal processing and compression for realistic media applications such as UHD, 3D, VR, HDR, and Hologram.

Haechul Choi, Hanbat National University, South Korea

Haechul Choi received his B.S. in electronics engineering from Kyungpook National University, Daegu, Korea, in 1997, and his M.S. and Ph.D. in electrical engineering from the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 1999 and 2004, respectively. He is a professor in Information and Communication Engineering at Hanbat National University, Daejeon, Korea. From 2004 to 2010, he was a Senior Member of the Research Staff in the Broadcasting Media Research Group of the Electronics and Telecommunications Research Institute (ETRI). His current research interests include image processing, video coding, and video transmission.


Luma Issa Abdul-Kreem and Hussam K. Adjul-Ameer. Object tracking using motion flow projection for pan-tilt configuration. volume 10, page 4687. Insitute of Advanced Engineeering and Science, 2020.

Cemal Aker and Sinan Kalkan. Using deep networks for drone detection. In 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1–6. IEEE, 2017.

J Athanesious and P Suresh. Implementation and comparison of kernel and silhouette based object tracking. International Journal of Advanced Research in Computer Engineering & Technology, pages 1298–1303, 2013.

Boris Babenko, Ming-Hsuan Yang, and Serge Belongie. Visual tracking with online multiple instance learning. In 2009 IEEE conference on computer vision and pattern recognition, pages 983–990. IEEE, 2009.

Rizki Yusliana Bakti, Intan Sari Areni, A Ais Prayogi, et al. Vehicle detection and tracking using gaussian mixture model and kalman filter. In 2016 International Conference on Computational Intelligence and Cybernetics, pages 115–119. IEEE, 2016.

Luca Bertinetto, Jack Valmadre, Stuart Golodetz, Ondrej Miksik, and Philip HS Torr. Staple: Complementary learners for real-time tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1401–1409, 2016.

Jean-Yves Bouguet et al. Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corporation, 5(1–10):4, 2001.

PJ Burt, JR Bergen, Rajesh Hingorani, R Kolczynski, WA Lee, A Leung, J Lubin, and H Shvayster. Object tracking with a moving camera. In [1989] Proceedings. Workshop on Visual Motion, pages 2–12. IEEE, 1989.

Dorin Comaniciu, Visvanathan Ramesh, and Peter Meer. Real-time tracking of non-rigid objects using mean shift. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662), volume 2, pages 142–149. IEEE, 2000.

Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303–338, 2010.

Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion. In Scandinavian conference on Image analysis, pages 363–370. Springer, 2003.

Helmut Grabner, Michael Grabner, and Horst Bischof. Real-time tracking via on-line boosting. In Bmvc, volume 1, page 6, 2006.

Kun Zheng Guangmin Sun, Junjie Zhang and Xiaohui Fu. Eye tracking and roi detection within a computer screen using a monocular camera. Journal of Web Engineering, 19(7–8):1117–1146, 2020.

João F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. High-speed tracking with kernelized correlation filters. IEEE transactions on pattern analysis and machine intelligence, 37(3):583–596, 2014.

Berthold KP Horn and Brian G Schunck. Determining optical flow. In Techniques and Applications of Image Understanding, volume 281, pages 319–331. International Society for Optics and Photonics, 1981.

Björn Johansson. Derivation of lucas-kanade tracker. In other words, 500:7, 2007.

Zdenek Kalal, Krystian Mikolajczyk, and Jiri Matas. Forward-backward error: Automatic detection of tracking failures. In 2010 20th International Conference on Pattern Recognition, pages 2756–2759. IEEE, 2010.

Man-Hyung Lee and Jong-Hwa Kim. Mathematical modelling of moving target and development of real time tracking method using kalman filter. The Korean Institute of Electrical Engineers, pages 765–769, 1987.

Xin Li, Kejun Wang, Wei Wang, and Yang Li. A multiple object tracking method using kalman filter. In The 2010 IEEE international conference on information and automation, pages 1862–1866. IEEE, 2010.

Bruce D Lucas, Takeo Kanade, et al. An iterative image registration technique with an application to stereo vision. 1981.

Deepika Punj Manisha Mudgal and Anuradha Pillai. Suspicious action detection in intelligent surveillance system using action attribute modelling. Journal of Web Engineering, 20(1):129–146, 2021.

Roberto Opromolla, Giancarmine Fasano, and Domenico Accardo. A vision-based approach to uav detection and tracking in cooperative applications. Sensors, 18(10):3391, 2018.

Divyani Prajapati and Hiren J Galiyawala. A review on moving object detection and tracking. International Journal of Computer Application, 5(3):168–175, 2015.

Jianbo Shi et al. Good features to track. In 1994 Proceedings of IEEE conference on computer vision and pattern recognition, pages 593–600. IEEE, 1994.

Arnold WM Smeulders, Dung M Chu, Rita Cucchiara, Simone Calderara, Afshin Dehghan, and Mubarak Shah. Visual tracking: An experimental survey. IEEE transactions on pattern analysis and machine intelligence, 36(7):1442–1468, 2013.

Galandaru Swalaganata, Yessi Affriyenni, et al. Moving object tracking using hybrid method. In 2018 International Conference on Information and Communications Technology (ICOIACT), pages 607–611. IEEE, 2018.

Amy R Wagoner, Daniel K Schrader, and Eric T Matson. Survey on detection and tracking of uavs using computer vision. In 2017 First IEEE International Conference on Robotic Computing (IRC), pages 320–325. IEEE, 2017.

Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Online object tracking: A benchmark. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2411–2418, 2013.

Sheldon Xu and Anthony Chang. Robust object tracking using kalman filters with dynamic covariance. Cornell University, pages 1–5, 2014.

Alper Yilmaz, Omar Javed, and Mubarak Shah. Object tracking: A survey. Acm computing surveys (CSUR), 38(4):13–es, 2006.

Tianzhu Zhang, Changsheng Xu, and Ming-Hsuan Yang. Multi-task correlation particle filter for robust object tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4335–4343, 2017.