Trustworthy Artificial Intelligence and Automatic Morse Code Based Communication Recognition with Eye Tracking

Authors

  • Krishnakanth Medichalam School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India
  • V. Vijayarajan School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India
  • V. Vinoth Kumar School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore 632014, India https://orcid.org/0000-0003-1070-3212
  • I. Manimozhi Iyer Department of Computer Science and Engineering, East Point College of Engineering and Technology, India
  • Yaswanth Kumar Vanukuri School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore 632014, India
  • V. B. Surya Prasath Department of Electrical Engineering and Computer Science, University of Cincinnati, OH 45221 USA
  • B. Swapna Department of ECE, Dr MGR Educational and Research Institute, Chennai-95, India

DOI:

https://doi.org/10.13052/jmm1550-4646.1964

Keywords:

Morse code, communication, blinks, gestures, Image Processing, face Detection, eye Detection

Abstract

Morse code is one of the oldest communication techniques and used in telecommunication systems. Morse code can be transmitted as a visual signal by using reflections or with the help of flashlights, but it can also be used as a non-detectable form of communication by using the tapping of fingers or even blinking of eyes. In this paper, we develop a computer vision based approach that automatically characterizes the characters conveyed wherein a person can communicate to system or another person through Morse code with eye gestures. We can decode this visual eye tracking based language with the help of our automatic computer vision driven method. Our approach uses a normal webcam to detect the gestures made by the eyes and are interpreted as dots and dashes. These dots and dashes are used to represent the Morse code-based words. Image processing techniques-based blink and pupil detectors are employed. Blink detector helps us to detect a blink and the time that took for each blink. A blink that takes 2 to 4 seconds is acknowledged as a dot whereas a blink that takes more than 4 seconds is represented as a dash. The pupil detector helps us to detect the movement of the pupils, and if pupils move towards right with respect to a person then it is acknowledged as next letter and if the pupils are moved towards left with respect to a person then it is acknowledged as next word. In this way, we decode the Morse code which will be communicated using eyes and establish a non-detectable communication between a person and an automatic system. Our experimental results on an unconstrained visual scene with preliminary greeting words indicate the promise of an automatic eye tracking based system with success rate of 98.25% that can be of use in non-verbal communications.

Downloads

Download data is not yet available.

Author Biographies

Krishnakanth Medichalam, School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India

Krishnakanth Medichalam completed his B.Tech in Computer Science and Engineering with Specialization in Bioinformatics from Vellore Institute of Technology. He is currently working as UI Developer in Tata Consultancy Services. His current research interests are Image processing, Modern Web Development and Natural Language Processing.

V. Vijayarajan, School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India

V. Vijayarajan received his B.E., Computer Science and Engineering from Madras University in 2001, M.E., Computer Science and Engineering from Anna University in 2007, and Ph.D., in Computer Science and Engineering from Vellore Institute of Technology in 2016. He is currently working as Associate Professor in School of Computer Science and Engineering, VIT University, Vellore, India. He has published 45+ articles in peer reviewed Journals which include reputed Conferences. His current research interests are Image Retrieval, Ontological Engineering, Operating Systems, Formal Languages and Automata Theory, Wireless Sensor Networks, Mobile Cloud Computing, Machine Learning and Deep Learning.

V. Vinoth Kumar, School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore 632014, India

V. Vinoth Kumar is an Associate Professor at School of Information Technology and Engineering, VIT University, Tamilnadu, India. He is currently being as an Adjunct Professor with School of Computer Science, Taylor’s University, Malaysia. He is having more than 12 years of teaching experience. He has organized various international conferences and delivered keynote addresses. He has published more than 60 research papers in various peer-reviewed journals and conferences. He is a Life Member of ISTE and a professional member of IEEE. He is the Associate Editor of International Journal of e-Collaboration (IJeC), International Journal of Pervasive Computing and Communications (IJPCC) and Editorial member of various journals. His current research interests include Wireless Networks, Internet of Things, machine learning and Big Data Applications. He has filled 9 patents in various fields of research.

I. Manimozhi Iyer, Department of Computer Science and Engineering, East Point College of Engineering and Technology, India

I. Manimozhi Iyer, currently working as an Associate Professor in the Department of Computer Science and Engineering, East Point College of Engineering and Technology, India. She has published more than 25 research articles in various journals. Her research interest in the fiels of Algorithms, Artificial Intelligence and Artificial Neural networks.

Yaswanth Kumar Vanukuri, School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore 632014, India

Yaswanth Kumar Vanukuri completed his B.Tech in Computer Science and Engineering from Vellore Institute of Technology. He is currently working as Sr.Tech Associate in Bank of America. His current research interests are Image processing and Machine Learning.

V. B. Surya Prasath, Department of Electrical Engineering and Computer Science, University of Cincinnati, OH 45221 USA

V. B. Surya Prasath, is a mathematician with expertise in the application areas of image processing, computer vision and machine learning. He received his PhD in mathematics from the Indian Institute of Technology Madras, India in 2009. He has been a postdoctoral fellow at the Department of Mathematics, University of Coimbra, Portugal, for two years from 2010 to 2011. From 2012 to 2015 he was with the Computational Imaging and VisAnalysis (CIVA) Lab at the University of Missouri, USA as a postdoctoral fellow, and from 2016 to 2017 as an assistant research professor. He is currently an assistant professor in the Division of Biomedical Informatics at the Cincinnati Children’s Hospital Medical Center, and at the Departments of Biomedical Informatics, Electrical Engineering and Computer Science, University of Cincinnati from 2018. He had summer fellowships/visits at Kitware Inc. NY, USA, The Fields Institute, Canada, and IPAM, University of California Los Angeles (UCLA), USA. His main research interests include nonlinear PDEs, regularization methods, inverse and ill-posed problems, variational, PDE based image processing, and computer vision with applications in remote sensing, biomedical imaging domains. His current research focuses are in data science, and bioimage informatics with machine learning techniques.

B. Swapna, Department of ECE, Dr MGR Educational and Research Institute, Chennai-95, India

B. Swapna, is currently working as Assistant Professor, ECE Department in Dr. M.G.R. Educational and Research Institute (Deemed to be University) Chennai, India. She has completed her ph.D in Internet of things, Master’s degree with honors in Applied Electronics secured Anna University 14th rank from SBC Engineering College, Arani and Bachelor’s degree in Electronics and Communication Engineering from from SBC Engineering College, Arani, India. Her areas of interest include VLSI, Embedded systems, IoT and Electronic circuits.

References

Al-Btoush, A.I., Abbadi, M.A., Hassanat, A.B., Tarawneh, A.S., Hasanat, A., Prasath, V.S.: New features for eye-tracking systems: Preliminary results. In: 2019 10th International Conference on Information and Communication Systems (ICICS), pp. 179–184. IEEE (2019). DOI: 10.1109/IACS.2019.8809129.

Bakde, N., Thakare, A.: Morse code decoder using a pic microcontroller. International Journal of Science, Engineering and Technology Research (IJSETR) 1(5), 200–205 (2012).

Bidani, S., Priya, R.P., Vijayarajan, V., Prasath, V.: Automatic body mass index detection using correlation of face visual cues. Technology and Health Care 28(1), 107–112 (2020). DOI: 10.3233/THC-191850.

Cheng, X., Zhao, X., Zhou, J., Zhang, H.: Eye localizaion method based on contour detection and d–s evidence theory. Signal, Image and Video Processing 12(4), 599–606 (2018). DOI: 10.1007/s11760-017-1165-9.

Fathi, A., Abdali-Mohammadi, F.: Camera-based eye blinks pattern detection for intelligent mouse. Signal, Image And Video Processing 9(8), 1907–1916 (2015). DOI: 10.1007/s11760-014-0680-1.

Ka, H., Simpson, R.: Effectiveness of morse code as an alternative control method for powered wheelchair navigation. In: Proceedings of the Annual Conference on Rehabilitation Engineering. RESNA (2012).

Venkatesan, V.K., Ramakrishna, M.T., Izonin, I., Tkachenko, R., Havryliuk, M. Efficient Data Preprocessing with Ensemble Machine Learning Technique for the Early Detection of Chronic Kidney Disease. Appl. Sci. 2023, 13, 2885. https://doi.org/10.3390/app13052885.

Devarajan, D., Alex, D. S., Mahesh, T. R., Kumar, V. V., Aluvalu, R., Maheswari, V. U., and Shitharth, S. (2022). Cervical Cancer Diagnosis Using Intelligent Living Behavior of Artificial Jellyfish Optimized With Artificial Neural Network. IEEE Access, 10, 126957–126968. https://doi.org/10.1109/access.2022.3221451.

Mukherjee, K., Chatterjee, D.: Augmentative and alternative communication device based on eye-blink detection and conversion to morse code to aid paralyzed individuals. In: International Conference on Communication, Information & Computing Technology, pp. 1–5. IEEE (2015). DOI: 10.1109/ICCICT.2015.7045754.

S. T. Ahmed, V. Kumar and J. Kim, “AITel: eHealth Augmented Intelligence based Telemedicine Resource Recommendation Framework for IoT devices in Smart cities,” in IEEE Internet of Things Journal, DOI: 10.1109/JIOT.2023.3243784.

Ravikumar, C., Dathi, M.: A fuzzy-logic based morse code entry system with a touch-pad interface for physically disabled persons. In: 2016 IEEE Annual India Conference (INDICON), pp. 1–5. IEEE (2016).

Sagonas, C., Antonakos, E., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: Database and results. Image and Vision Computing 47, 3–18 (2016). DOI: 10.1016/j.imavis.2016. 01.002.

Sapaico, L.R., Sato, M.: Analysis of vision-based text en- try using morse code generated by tongue gestures. In: 2011 4th International Conference on Human System Interactions, HSI 2011, pp. 158–164. IEEE (2011). DOI: 10.1109/HSI.2011.5937359.

Singh, H., Singh, J.: Real-time eye blink and wink detection for object selection in HCI systems. Journal on Multimodal User Interfaces 12(1), 55–65 (2018). DOI: 10.1007/s12193-018-0261-7.

Srividhya, G., Murali, S., Keerthana, A., Rubi, J.: Alter- native voice communication device using eye blink detection for people with speech disorders. International Journal of Recent Technology and Engineering 8(4), 12,541–12,543 (2019). DOI: 10.35940/ijrte.D5405.118419.

Viola, P., Jones, M.J.: Robust real-time face detection. International Journal of Computer Vision 57(2), 137–154 (2004). DOI: 10.1023/B:VISI.0000013087.49260.fb.

Babitha D., Jayasankar T., Sriram V.P., Sudhakar S., Prakash K.B., Speech emotion recognition using state-of-art learning algorithms 2020, International Journal of Advanced Trends in Computer Science and Engineering, 9(2), 1340–1345, DOI: 10.30534/ijatcse/2020/67922020.

Bharadwaj Y.S.S., Rajaram P., Sriram V.P., Sudhakar S., Prakash K.B. Effective handwritten digit recognition using deep convolution neural network, 2020, International Journal of Advanced Trends in Computer Science and Engineering, 9(2), 1335–1339, DOI: 10.30534/ijatcse/2020/66922020.

Kim, S. and Yoon, K. (2022). Deep and Lightweight Neural Network for Histopathological Image Classification. Journal of Mobile Multimedia, 18(06), 1913–1930. https://doi.org/10.13052/jmm1550-4646.18619.

Shevchuk, B., Ivakhiv, O., Zastavnyy, O., and Geraimchuk, M. (2023). Telemonitoring of Human Biomedical and Biomechanical Signals. Journal of Mobile Multimedia, 19(03), 877–896. https://doi.org/10.13052/jmm1550-4646.19310.

Downloads

Published

2023-10-14

How to Cite

Medichalam, K. ., Vijayarajan, V. ., Kumar, V. V. ., Iyer, I. M. ., Vanukuri, Y. K. ., Prasath, V. B. S. ., & Swapna, B. . (2023). Trustworthy Artificial Intelligence and Automatic Morse Code Based Communication Recognition with Eye Tracking. Journal of Mobile Multimedia, 19(06), 1439–1462. https://doi.org/10.13052/jmm1550-4646.1964

Issue

2023: Vol 19 Iss 6

Section

Federated trustworthy artificial intelligence for multimedia data

Most read articles by the same author(s)