Multimodal Driver Drowsiness Detection From Video Frames

Authors

  • Pritesh Kumar Singh Bharati Vidyapeeth’s College of Engineering, New Delhi, India
  • Archit Gupta Bharati Vidyapeeth’s College of Engineering, New Delhi, India
  • Mayank Upadhyay Bharati Vidyapeeth’s College of Engineering, New Delhi, India
  • Achin Jain Bharati Vidyapeeth’s College of Engineering, New Delhi, India
  • Manju Khari Jawaharlal Nehru University, Delhi School of Computer and Systems Sciences, New Delhi, India
  • Puneet Singh Lamba VIPS-TC, School of Engineering & Technology, New Delhi, India

DOI:

https://doi.org/10.13052/jmm1550-4646.19210

Keywords:

Eye aspect ratio (EAR), mouth aspect ratio (MAR), facial landmarks, Drowsiness

Abstract

Fatigue leads to tiredness, exhaustion, and sleepiness. Driving in fatigue conditions is considered dangerous and can cause serious road accidents. According to reports about 25% of road accidents are due to driver drowsiness. The main reason behind drowsiness is fatigue. While driving continuously on long trips, drivers feel sleepy. In this paper, we proposed a novel approach that is efficient enough to detect driver drowsiness accurately. An intelligent system, that can quickly and precisely determine whether the driver is feeling drowsiness or not during driving and can also generate a warning in real-time scenarios is implemented. Thus, resulting in reducing the number of accidents that take place due to the drowsiness of the drivers as well as the death rate. In this paper, drowsiness is detected by observing facial features such as Eyes and Mouth.

Downloads

Download data is not yet available.

Author Biographies

Pritesh Kumar Singh, Bharati Vidyapeeth’s College of Engineering, New Delhi, India

Pritesh kumar Singh received Bachelor of Technology (B.Tech) in Information Technology from Bharati Vidyapeeth’s College Of Engineering (New Delhi) affiliated to Guru Gobind Singh Indraprastha University in 2021. He is currently working as Associate Software Engineer in Nagarro.

Archit Gupta, Bharati Vidyapeeth’s College of Engineering, New Delhi, India

Archit Gupta received Bachelor of Technology (B.Tech) in Information Technology from Bharati Vidyapeeth’s College Of Engineering (New Delhi) affiliated to Guru Gobind Singh Indraprastha University in 2021. He is currently working as Associate Software Engineer in Nagarro.

Mayank Upadhyay, Bharati Vidyapeeth’s College of Engineering, New Delhi, India

Mayank Upadhyay received the bachelor degree (B-tech) in Information Technology from Bharati Vidyapeeth’s College Of Engineering, New Delhi, Guru Gobind Singh Indraprastha University in 2021. He is continuously working in the field of Data Science and AI which includes neural networks (ANN,RNN,CNN), openCV,Tensorflow,NLP etc.

Achin Jain, Bharati Vidyapeeth’s College of Engineering, New Delhi, India

Achin Jain is research scholar of University School of Information, Communication and Technology, GGSIPU, Sector 16 C, Dwarka, Delhi. He is currently associated as an Assistant Professor with Information Technology Department of Bharati Vidyapeeth’s College of Engineering. His main research areas are Sentiment Classification and Machine Learning using NLP techniques.

Manju Khari, Jawaharlal Nehru University, Delhi School of Computer and Systems Sciences, New Delhi, India

Manju Khari is an Associate Professor in Jawaharlal Nehru University, Delhi, India School of computer and Systems Sciences. She is also the Professor- In-charge of the IT Services of the Institute and has experience of more than twelve years in Network Planning & Management. She holds a Ph.D. in Computer Science & Engineering from National Institute of Technology Patna and she received her master’s degree in Information Security from Ambedkar Institute of Advanced Communication Technology and Research, formally this institute is known as Ambedkar Institute Of Technology affiliated with Guru Gobind Singh Indraprastha University, Delhi, India. Her research interests are software testing, information security, optimization, Image processing and machine learning. She has 70 published papers in refereed National/International Journals & Conferences (viz. IEEE, ACM, Springer, Inderscience, and Elsevier) and 10+ edited books from reputed publishers. She is also co-author of two books published by NCERT of Secondary and senior Secondary School.

Puneet Singh Lamba, VIPS-TC, School of Engineering & Technology, New Delhi, India

Puneet Singh Lamba is currently working as Assistant Professor in Vivekananda Institute of Professional Studies-Technical Campus, School of Engineering and Technology. He received his PhD degree in Information Technology from USIC&T, GGSIPU. He obtained his master’s degree in Information Technology from USIC&T, GGSIPU. He completed his bachelor’s degree in Information Technology from USIC&T, GGSIPU. He has a total experience of 13 years. He has multiple publications in the area of Information Retrieval, Artificial Intelligence, and Image Processing. He has Qualified UGC NET(2015) and GATE (2010) Exam.

References

S. Sangle, B. Rathore, R. Rathod, A. Yadav, and A. Yadav, “Real-Time Drowsiness Detection System,” IOSR Journal of Computer Engineering (IOSR-JCE), pp. 87–92, 2018

Das, K., & Behera, R. N. (2017). A survey on machine learning: concept, algorithms and applications. International Journal of Innovative Research in Computer and Communication Engineering, 5(2), 1301–1309.

Alioua, N., Amine, A., Rziza, M., & Aboutajdine, D. (2011, August). Driver’s fatigue and drowsiness detection to reduce traffic accidents on road. In International Conference on Computer Analysis of Images and Patterns (pp. 397–404). Springer, Berlin, Heidelberg..

Mehta, S., Dadhich, S., Gumber, S., & Jadhav Bhatt, A. (2019, February). Real-time driver drowsiness detection system using eye aspect ratio and eye closure ratio. In Proceedings of international conference on sustainable computing in science, technology and management (SUSCOM), Amity University Rajasthan, Jaipur-India.

Soukupova, T., & Cech, J. (2016, February). Eye blink detection using facial landmarks. In 21st computer vision winter workshop, Rimske Toplice, Slovenia.

Fuletra, J. D., & Bosamiya, D. (2013). A survey on drivers drowsiness detection techniques. International Journal on Recent and Innovation Trends in Computing and Communication, 1(11), 816–819.

Mehta, S., Dadhich, S., Gumber, S., & Jadhav Bhatt, A. (2019, February). Real-time driver drowsiness detection system using eye aspect ratio and eye closure ratio. In Proceedings of international conference on sustainable computing in science, technology and management (SUSCOM), Amity University Rajasthan, Jaipur-India.

Podder, S., & Roy, S. (2013). Driver’s drowsiness detection using eye status to improve the road safety. International Journal of Innovative Research in Computer and Communication Engineering, 1(7), 1490–1497.

Omidi, F., & Saraji, G. N. (2016). Non-intrusive Methods used to Determine the Driver Drowsiness: Narrative Review Articles. International Journal of Occupational Hygiene, 8(4), 186–191.

Nagargoje, S. S., & Shilvant, D. S. (2015). Drowsiness detection system for Car assisted driver using image processing. International Journal of Electrical and Electronics Research ISSN, 3(4), 175–179.

Saini, V., & Saini, R. (2014). Driver drowsiness detection system and techniques: a review. International Journal of Computer Science and Information Technologies, 5(3), 4245–4249.

K. C. Patel, S. A. Khan, and V. N. Patil, “Real-Time Driver Drowsiness Detection System Based on Visual Information,” International Journal of Engineering Science and Computing, Volume 8, No. 3, pp. 16200–16203

Garcia, I., Bronte, S., Bergasa, L. M., Almazán, J., & Yebes, J. (2012, June). Vision-based drowsiness detector for real driving conditions. In 2012 IEEE Intelligent Vehicles Symposium (pp. 618–623). IEEE.

Park, S., Pan, F., Kang, S., & Yoo, C. D. (2016, November). Driver drowsiness detection system based on feature representation learning using various deep networks. In Asian Conference on Computer Vision (pp. 154–164). Springer, Cham.

Chisty, J. G. (2015). A Review: Driver drowsiness detection system. IJCST, 3(4), 243–252.

Bhor, R., Mahajan, P., & Kumbhar, H. V. (2015). Survey on driver’s drowsiness detection system. International Journal of Computer Applications, 132(5), 16–19.

Deepa, K. B., Chaitra, M., Sharma, A. K., Sreedhar, V. S., & Kumar, H. K. (2015). Accident prevention by eye blinking sensor and alcohol detector. International Journal of Engineering Research, 4(7), 351–354.

Tejasweenimusale, Prof B, H. Pansambal, “Real Time Driver Drowsiness Detection System using Image Processing”, IJREAM, Vol. 02, Issue 08, 2016.

Triyanti, V., & Iridiastadi, H. (2017, December). Challenges in detecting drowsiness based on driver’s behavior. In IOP Conference Series: Materials Science and Engineering (Vol. 277, No. 1, p. 012042). IOP Publishing.

Anandan, P., & Sabeenian, R. S. (2013). Image Compression Techniques using Curvelet, Contourlet, Ridgelet and Wavelet Transforms–A Review. Biometrics and Bioinformatics, 5(7), 267–270.

Nguyen, Q. N., Tho, L. T. A., Van, T. V., Yu, H., & Thang, N. D. (2017, June). Visual Based Drowsiness Detection Using Facial Features. In International Conference on the Development of Biomedical Engineering in Vietnam (pp. 723–727). Springer, Singapore.

M. Hemamalini, P. Muhilan“Accident prevention using eye blink sensor”, Asia Pacific Journal of Research, vol. 1, Issue L11, 2017.

El-Shazly, E. H., Abdelwahab, M. M., Shimada, A., & Taniguchi, R. I. (2016, October). Real time algorithm for efficient HCI employing features obtained from MYO sensor. In 2016 IEEE 59th International Midwest Symposium on Circuits and Systems (MWSCAS) (pp. 1–4). IEEE.

Xu, Z., Qiu, X., & He, J. (2016, August). A novel multimedia human-computer interaction (HCI) system based on kinect and depth image understanding. In 2016 International Conference on Inventive Computation Technologies (ICICT) (Vol. 3, pp. 1–6). IEEE.

Smirnov, A., Kashevnik, A., Lashkov, I., Baraniuc, O., & Parfenov, V. (2016, April). Smartphone-based identification of dangerous driving situations: algorithms and implementation. In 2016 18th Conference of Open Innovations Association and Seminar on Information Security and Protection of Information Technology (FRUCT-ISPIT) (pp. 306–313). IEEE.

Chang, K., Oh, B. H., & Hong, K. S. (2014, January). An implementation of smartphone-based driver assistance system using front and rear camera. In 2014 IEEE International Conference on Consumer Electronics (ICCE) (pp. 280–281). IEEE.

Xu, L., Li, S., Bian, K., Zhao, T., & Yan, W. (2014, February). Sober-Drive: A smartphone-assisted drowsy driving detection system. In 2014 International conference on computing, networking and communications (ICNC) (pp. 398–402). IEEE.

Singh, P. K., Upadhyay, M., Gupta, A., & Lamba, P. S. (2021). CNN-Based Driver Drowsiness Detection System. In Concepts and Real-Time Applications of Deep Learning (pp. 153–166). Springer, Cham.

Ambekar, S. N., Korde, M. R., & Patil, S. R. (2016). Driver drowsiness detection system. Int J Sci Technol Manage Res, 1(9).

Singh, V., Elamvazuthi, I., Jeoti, V., & George, J. (2014, June). 3D reconstruction of ATFL ligament using ultrasound images. In 2014 5th International Conference on Intelligent and Advanced Systems (ICIAS) (pp. 1–5). IEEE.

Bappaditya Mandal, Liyuan Liyuan Li, Gang Sam Wang, and JieLin “Towards detection of bus driver fatigue based on robust visual analysis of eye state”, IEEE transaction on intelligent transportation systems, 2016.

Marimuthu, R., Suresh, A., Alamelu, M., & Kanagaraj, S. (2017). Driver fatigue detection using image processing and accident prevention. Int J Pure Appl Math, 116(11), 91–99.

Hwang, T., Kim, M., Hong, S., & Park, K. S. (2016, August). Driver drowsiness detection using the in-ear EEG. In 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 4646–4649). IEEE.

Joyce, G., Lilley, M., Barker, T., & Jefferies, A. (2016, July). Mobile application tutorials: perception of usefulness from an HCI expert perspective. In International Conference on Human-Computer Interaction (pp. 302–308). Springer, Cham.

Kang, P., Wei, Y., & Wei, Z. (2017, May). Control system for granary ventilation based on embedded networking and Qt technology. In 2017 29th Chinese Control And Decision Conference (CCDC) (pp. 2275–2280). IEEE.

Gupta, I., Garg, N., Aggarwal, A., Nepalia, N., & Verma, B. (2018, August). Real-time driver’s drowsiness monitoring based on dynamically varying threshold. In 2018 Eleventh International Conference on Contemporary Computing (IC3) (pp. 1–6). IEEE.

Downloads

Published

2022-11-15

How to Cite

Singh, P. K. ., Gupta, A. ., Upadhyay, M. ., Jain, A. ., Khari, M. ., & Lamba, P. S. . (2022). Multimodal Driver Drowsiness Detection From Video Frames. Journal of Mobile Multimedia, 19(02), 567–586. https://doi.org/10.13052/jmm1550-4646.19210

Issue

2023: Vol 19 Iss 2

Section

Neural Networks for Intelligent Multimedia Signal Processing and Analysis