Enhancing Gesture-Controlled Virtual Mouse and Virtual Keyboard Using AI Techniques

Authors

  • Jayasri Kotti Department of Information Technology, GMR Institute of Technology, Rajam, Andhra Pradesh, India
  • B. Padmaja Department of Information Technology, GMR Institute of Technology, Rajam, Andhra Pradesh, India
  • D. Deepa School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, Tamilnadu, India

DOI:

https://doi.org/10.13052/jmm1550-4646.2029

Keywords:

Hand gestures, OpenCV, media pipe, virtual mouse, virtual keyboard, virtual assistant

Abstract

Artificial Intelligence has become an essential part of modern technology. Although computer technology is advanced, it can still be improved to make it more user-friendly. One way to do this is to replace touchscreen desktops with a virtual mouse and keyboard. This can reduce the need for gadgets and enhance human-computer interaction. During the COVID-19 pandemic, reducing human intervention and dependency on devices has been critical in controlling the spread of the virus. This is where a battery-powered or Bluetooth mouse, powered by virtual reality technology, can be helpful. The virtual mouse is created using OpenCV and virtual reality technology, with the proposed system utilizing advanced tools such as MediaPipe and Python. The MediaPipe library is particularly useful in artificial intelligence projects, as it enhances the efficiency of the model. The system is an AI-based mouse and keyboard that can be controlled using hand gestures. The user interacts with the system through the camera output displayed on the screen, while the webcam serves as an input device. Python and OpenCV tools are used for implementation, making it applicable in pandemic situations and smart teaching systems. The proposed system works on Enhancing gesture Controlled Virtual Mouse and Virtual Keyboard through Virtual Assistant using AI Techniques.

Downloads

Download data is not yet available.

Author Biographies

Jayasri Kotti, Department of Information Technology, GMR Institute of Technology, Rajam, Andhra Pradesh, India

Jayasri Kotti, received her doctorate from Andhra University, Visakhapatnam, M.Tech (Information Technology) from Andhra University, Visakhapatnam, AP, India. She received a Gold Medal for Best Thesis from Andhra University. She worked as Dean R & D and Professor in the Department of Computer Science and Engineering, Vignan’s Institute of Engineering for Women, Visakhapatnam, Andhra Pradesh, India. She has 15 years of Teaching experience. She has more than 30 publications in reputed journals and guided UG, PG, and PhD thesis. She also published 5 patents, 2 books, and 6 book chapters. She got the best faculty award and best researcher award at the college level. Her research areas are Software Engineering, Machine Learning, Artificial Intelligence and Internet of Things. At present, she is working at GMR Institute of Technology, Rajam, Andhra Pradesh, and doing active research work.

B. Padmaja, Department of Information Technology, GMR Institute of Technology, Rajam, Andhra Pradesh, India

B. Padmaja is an Assistant Professor in the Department of Information Technology at GMR Institute of Technology (GMRIT). She is currently pursuing a part-time Ph.D. at Andhra University alongside her teaching responsibilities. With her academic background and ongoing research, Ms. B. Padmaja is dedicated to contributing to the field of Information Technology through her teaching, research, and academic pursuits. Her research interests primarily revolve around Artificial Intelligence (AI) and Machine Learning (ML), showcasing her dedication to advancing these fields through academic and research contributions. With a robust background in these areas, Ms. B. Padmaja is committed to making significant strides in AI and ML through her academic and research endeavors.

D. Deepa, School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, Tamilnadu, India

D. Deepa is currently working as an Assistant professor in Vellore Institute of Technology, Vellore, Tamilnadu, India. She completed her master degree in Computer Science and Engineering at Kongu Engineering College, Anna University, in 2011 and completed her PhD in Information and Communication and Engineering in Anna university, Chennai in 2024. Her major research interests are Natural Language Processing and Deep Learning Models.

References

Rustagi, Dhruv and Maindola et al., ‘Virtual Control Using Hand-Tracking’, International Journal for Modern Trends in Science and Technology. 8. 26–31. doi: 10.46501/IJMTST0801005, 2022.

Shukla, A., Katiyar, D. D., and Goel, M. G, ‘Gesture Recognition-based AI Virtual Mouse’. International Journal for Research in Applied Science and Engineering Technology, 10(3), 1583–1588, 2022, March 31, https://doi.org/10.22214/ijraset.2022.40937.

Shriram, S., Nagaraj, Bowthiya, Jaya, J., Shankar, S. and Ajay, P, ‘Deep Learning-Based Real-Time AI Virtual Mouse System Using Computer Vision to Avoid COVID-19 Spread’, Journal of Healthcare Engineering, 2021, doi: 10.1155/2021/8133076.

Shivanand, Nirmala. “AI And Ml Based Gesture Controlled Virtual Mouse Actions System-A Novel Approach.”

V. S. Sangtani, A. Porwal, A. Kumar, A. Sharma, and A. Kaushik, ‘Artificial Intelligence Virtual Mouse using Hand Gesture’, IJMDES, vol. 2, no. 5, pp. 26–30, May 2023.

Kavitha R, Janasruthi S U et al., ‘Hand gesture controlled virtual mouse using artifical intelligence’, ijariie, vol-9, issue-2, ISSN(O)-2395-4396, 2023.

D. Saritha P. Venkata Siva Prasad et al., ‘Gesture Controlled AI Virtual Mouse System Using Computer Vision’, International Journal of Research in Engineering, IT and Social Sciences, ISSN 2250-058812(7), 134–138, July 2022.

Kathar, Shashank, Shubham Jagtap et al., ‘AI Virtual Mouse’, IJAEM.net.

Kharbanda, K., and Sachdeva, U, ‘Gesture Controlled Virtual Mouse Using Artificial Intelligence’, International Research Journal of Modernization in Engineering Technology and Science, Volume:05/Issue:01/January-2023.

C. D. Sai Nikhil, C. U. Someswara Rao, E. Brumancia, K. Indira, T. Anandhi and P. Ajitha, ‘Finger Recognition and Gesture based Virtual Keyboard’, 2020 5th International Conference on Communication and Electronics Systems (ICCES), Coimbatore, India, 2020, pp. 1321–1324, doi: 10.1109/ICCES48766.2020.9137889.

Rahim, Md Abdur, Jungpil Shin, and Md Rashedul Islam, ‘Hand gesture recognition-based non-touch character writing system on a virtual keyboard’, Multimedia Tools and Applications 79, no. 17–18 (2020): 11813–11836.

S. Subhash, P. N. Srivatsa, S. Siddesh, A. Ullas and B. Santhosh, ‘Artificial Intelligence-based Voice Assistant’, 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), London, UK, 2020, pp. 593–596, doi: 10.1109/WorldS450073.2020.9210344.

S. Malodia, N. Islam, P. Kaur and A. Dhir, ‘Why Do People Use Artificial Intelligence (AI)-Enabled Voice Assistants’, IEEE Transactions on Engineering Management, doi: 10.1109/TEM.2021.3117884.

R. S. Sai Dinesh, R. Surendran, D. Kathirvelan and V. Logesh, “Artificial Intelligence based Vision and Voice Assistant,” 2022 International Conference on Electronics and Renewable Systems (ICEARS), Tuticorin, India, 2022, pp. 1478–1483, doi: 10.1109/ICEARS53579.2022.9751819.

Manojkumar, M. P. K., Patil, A., Shinde, S., Patra, S., and Patil, S, ‘AI-Based Virtual Assistant Using Python: A Systematic Review’, IJRASET, ISSN : 2321-9653, 2023

G. Preethi, Abishek. K, Thiruppugal S, Vishwaa D A, 2022, ‘Voice Assistant using Artificial Intelligence’, International Journal of Engineering Research & Technology (Ijert) Volume 11, Issue 05 (May 2022).

S. R. Chowdhury, S. Pathak and M. D. A. Praveena, ‘Gesture Recognition Based Virtual Mouse and Keyboard’, 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI) (48184), Tirunelveli, India, 2020, pp. 585–589, doi: 10.1109/ICOEI48184.2020. 9143016.

Bhakare, M. A., Rathod, M. A., Kolhe, M. A., Shardul, M. K., and Kumar, N, ‘Implementation of Gesture Recognition Stystem With Virtual Mouse And Keyboard Using Haar Cascade Algorithm’, International Research Journal of Modernization in Engineering Technology and Science’, Volume:05/Issue:06/June-2023.

Pardeshi, S., Jagtap, S., Kathar, S., Giri, S., and Kapare, S, ‘AI Virtual Mouse and Keyboard’, IJAEM.net.

J, Prithvi, et al. ‘Gesture Controlled Virtual Mouse with Voice Automation’, www.ijert.org/research/gesture-controlled-virtual-mouse-with-voice-automation-IJERTV12IS040131.pdf. Accessed 7 July 2023.

R, Likitha, et al. ‘AI Virtual Mouse System Using Hand Gestures and Voice Assistant’, IJERAL, www.ijera.com/papers/vol12no12/S1212132138.pdf. Accessed 7 July 2023.

Khushi Patel, Snehal Solaunde, Shivani Bhong, Prof. Sairabanu Pansare, ‘Virtual Mouse Using Hand Gesture and Voice Assistant’, International Journal of Innovative Research in Technology (IJIRT), Volume 9, Issue 2, July 2022, ISSN: 2349-6002.

M. Strbo, ‘AI based Smart Teaching Process During The Covid-19 Pandemic’, 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS), Thoothukudi, India, 2020, pp. 402–406, doi: 10.1109/ICISS49785.2020.9315963.

G. Sziladi, T. Ujbanyi, J. Katona and A. Kovari, ‘The analysis of hand gesture-based cursor position control during solve an IT related task’, 2017 8th IEEE International Conference on Cognitive Info communications (CogInfoCom), Debrecen, Hungary, 2017, pp. 000413–000418, doi: 10.1109/CogInfoCom.2017.8268281.

Katona, J. A, ‘Review of Human–Computer Interaction and Virtual Reality Research Fields in Cognitive Info Communications’, Appl. Sci. 2021, 11, 2646. https://doi.org/10.3390/app11062646.

Downloads

Published

2024-03-29

How to Cite

Kotti, J. ., Padmaja, B., & Deepa, D. (2024). Enhancing Gesture-Controlled Virtual Mouse and Virtual Keyboard Using AI Techniques. Journal of Mobile Multimedia, 20(02), 437–494. https://doi.org/10.13052/jmm1550-4646.2029

Issue

2024: Vol 20 Iss 2

Section

Articles