AIoT Smart Eyewear with Real-time Object and Audio Recognition for Visually Impaired Users

Authors

  • Pritam Nanda Department of Computer Science & Engineering, Silicon University, Odisha, India
  • Soumya Ranjan Samal Silicon University, Odisha, India
  • Shuvabrata Bandopadhaya Banasthali Vidyapith, Rajasthan, India, Gurugram, India
  • Debi Prasad Pradhan Silicon University, Odisha, India
  • Antoni Ivanov Faculty of Telecommunications, Technical University of Sofia, Sofia, Bulgaria
  • Vladmir Poulkov Faculty of Telecommunications, Technical University of Sofia, Sofia, Bulgaria

DOI:

https://doi.org/10.13052/jmm1550-4646.2212

Keywords:

Assistive System, Roboflow, Tesseract OCR, TF-Luna LiDAR, Visually Impaired, VIuNI, YOLO

Abstract

The goal of the proposed smart eyewear tool is to assist visually impaired (VI) or blind individuals by providing object detection and real-time audio descriptions of their surrounding environment. This will help these users in navigating their surroundings safely and independently. This is achieved through the integration of artificial intelligence (AI) and Internet of Things (IoT) technologies into a compact, wearable system. The proposed work presents a real-time, artificial intelligence of things (AIoT) based assistive system designed by using an ESP32-CAM module for real-time object detection, integrated with a TF-Luna LiDAR sensor for distance measurement, and a Bluetooth-enabled neckband for audio feedback. The system uses YOLO for object detection and Roboflow for dataset preparation and training. It also includes features such as night-time navigation and an emergency alert for enhanced safety. Experimental results showed high performance in various lighting conditions, with object detection accuracy of 94.93%, a mean absolute error (MAE) of 0.34 cm, and a root mean square error (RMSE) of 0.44 cm in distance estimation. The SOS alert system responded with 100% accuracy in emergency situations.

Downloads

Download data is not yet available.

Author Biographies

Pritam Nanda, Department of Computer Science & Engineering, Silicon University, Odisha, India

Pritam Nanda is an Assistant Professor in the Faculty of Engineering and Technology at Sri Sri University, India. He is currently pursuing his Ph.D. at Silicon University, India. He received his B.Tech. degree in electronics and communication engineering from Biju Patnaik University of Technology (BPUT), India, in 2015, and subsequently completed his M.Tech. degree in computer science and engineering from BPUT, India, in 2024. His academic and research interests include cloud computing, Internet of Things (IoT), data analytics, artificial intelligence, and machine learning, with a particular focus on their applications in healthcare and smart systems. He is actively engaged in teaching, research, student mentoring, and academic coordination, and his professional activities reflect a strong commitment to innovation, institutional development, and technology-driven education.

Soumya Ranjan Samal, Silicon University, Odisha, India

Soumya Ranjan Samal received his Ph.D. degree in communication networks, faculty of telecommunications from Technical University of Sofia at Sofia, Bulgaria. He received his B.Tech. degree in electronics & instrumentation engineering from Biju Patnaik University of Technology, India in 2004. Soumya then went on to pursue his M.Eng. in computer science & engineering from the Utkal University of Bhubaneswar, India in 2009. He, as an Associate Professor in Silicon University, India, has acquired solid experience of about 18 years of teaching in communication engineering. Soumya also worked as a Project Engineer in Indian Institute of Technology, Bombay, India in 2005. His research areas of interest include, Interference management in 5G cellular networks, green communication movement to develop energy efficient solutions through antenna parameters and IoT.

Shuvabrata Bandopadhaya, Banasthali Vidyapith, Rajasthan, India, Gurugram, India

Shuvabrata Bandopadhaya is currently working as an associate professor in the department of electronics at the School of Physical Sciences, Banasthali Vidyapith, Rajasthan. He received his M.Tech. and Ph.D. degrees in communication systems specialisation from KIIT University, Bhubaneswar, India. He has nearly 20 years of experience in teaching and research at various reputed institutes and universities in India. His areas of research interest include wireless communication and networks, Internet of Things, and AI.

Debi Prasad Pradhan, Silicon University, Odisha, India

Debi Prasad Pradhan is currently working as a Technical Assistant at the IoT Lab and Industrial Control Lab at Silicon University, Bhubaneswar. He holds an M.Tech. degree in electronics and communication (2018) from CET Bhubaneswar, Odisha, and a B.Tech. in electronics and communication (2015). He has over 15 years of academic and mentoring experience at Silicon University, India. His research interests include AI-integrated wireless systems for IoT, optical sensor network design, and emerging technologies in industrial control, particularly PLC and DCS systems for Industry 5.0.

Antoni Ivanov, Faculty of Telecommunications, Technical University of Sofia, Sofia, Bulgaria

Antoni Ivanov received his Ph.D. degree in communication networks and systems from the Technical University of Sofia (TUS), Bulgaria. He holds a master’s degree in innovative communication technologies and entrepreneurship from TUS, and Aalborg University, Denmark in 2016. He is currently a postdoctoral researcher at the “Teleinfrastructure Lab”, Faculty of Telecommunications, TUS. His research interests include cognitive radio networks, adaptive algorithms for dynamic spectrum access, deep learning-based solutions for cognitive radio applications, volumetric spectrum occupancy assessment, and graph signal processing for resource allocation in current and future wireless networks.

Vladmir Poulkov, Faculty of Telecommunications, Technical University of Sofia, Sofia, Bulgaria

Vladimir Poulkov received his M.Sc. and Ph.D. degrees from the Technical University of Sofia (TUS), Sofia, Bulgaria. He has more than 30 years of teaching, research, and industrial experience in the field of telecommunications. He has successfully managed numerous industrial, engineering, R&D and educational projects. He has been Dean of the Faculty of the Telecommunications at TUS and Vice Chairman of the General Assembly of the European Telecommunications Standardization Institute (ETSI). Currently, he is the Head of the “Teleinfrastructure” R&D Laboratory at TUS and Chairman of Cluster for Digital Transformation and Innovation, Bulgaria. He is Fellow of the European Alliance for Innovation and a Senior IEEE Member. He has authored many scientific publications and is tutoring B.Sc., M.Sc., and Ph.D. courses in the field of information transmission theory and wireless access networks.

References

WHO, “Visual impairment and blindness,” August 2023.

M. Mashiata, B. Mondal, K. Deb, P. P. Paul, and M. R. Islam, “Towards assisting visually impaired individuals: A review on current status and future prospects,” Biosensors and Bioelectronics: X, vol. 12, p. 100265, 2022.

R. Tapu, B. Mocanu, and T. Zaharia, “Wearable assistive devices for visually impaired: A state-of-the-art survey,” Pattern Recognition Letters, vol. 137, pp. 37–52, 2020.

P. Kathiria, P. Parmar, D. Kothari, A. Patel, and P. Solanki, “Assistive systems for visually impaired people: A survey on current requirements and advancements,” Neurocomputing, vol. 606, p. 128284, 2024.

W. Elmannai and K. Elleithy, “A highly accurate and reliable data fusion framework for guiding the visually impaired,” IEEE Access, vol. 6, pp. 33029–33054, 2018.

M. Hersh and M. Johnson, “On modelling assistive technology systems–part i: Modelling framework,” Technology and Disability, vol. 20, pp. 193–215, 2008.

Z. Muhsin, R. Qahwaji, F. Ghanchi, S. Al-Alwani, and R. Al-Mufarji, “Review of substitutive assistive tools and technologies for people with visual impairments: recent advancements and prospects,” Journal of Multimodal User Interfaces, vol. 18, pp. 135–156, 2024.

I. P. Singh, A. Misra, A. Maurya, and A. Verma, “Technical review of smart assistive devices for visually impaired people,” in 2023 4th International Conference for Emerging Technology (INCET), (Belgaum, India), pp. 1–12, 2023.

A. Lavric, C. Beguni, E. Zadobrischi, A.-M. Căilean, and S.-A. Avătămăniței, “A comprehensive survey on emerging assistive technologies for visually impaired persons: Lighting the path with visible light communications and artificial intelligence innovations,” Sensors, vol. 24, no. 15, p. 4834, 2024.

K. Manjari, M. Verma, and G. Singal, “A survey on assistive technology for visually impaired,” Internet of Things, vol. 11, p. 100188, 2020.

R. Velázquez, “Wearable assistive devices for the blind,” in Wearable and Autonomous Biomedical Devices and Systems for Smart Environment (A. Lay-Ekuakille and S. Mukhopadhyay, eds.), vol. 75 of Lecture Notes in Electrical Engineering, Berlin, Heidelberg: Springer, 2010.

J. Brabyn, K. Seelman, and S. Panchang, “Aids for people who are blind or visually impaired,” in An introduction to rehabilitation engineering (R. Cooper, H. Ohnabe, and D. Hobson, eds.), pp. 287–313, Taylor & Francis, 2007.

M. H. Abidi, A. N. Siddiquee, H. Alkhalefah, and V. Srivastava, “A comprehensive review of navigation systems for visually impaired individuals,” Heliyon, vol. 10, no. 11, p. e31825, 2024.

Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,” Proceedings of the IEEE, vol. 111, no. 3, pp. 257–276, 2023.

K. Iqbal, S. S. Ali, Z. Sajid, M. Samad, L. Mubarak, and S. Ali, “Advancement in smart vision systems: A computer vision-based assistive system for visually impaired individuals,” VAWKUM Transactions on Computer Sciences, vol. 13, no. 1, pp. 244–257, 2025.

A. A., B. D. J., S. P. Mary, and D. U. Nandini, “Portable camera based identification system for visually impaired people,” in 2023 7th International Conference on Trends in Electronics and Informatics (ICOEI), (Tirunelveli, India), pp. 1444–1450, 2023.

H. Baskaran, R. L. M. Leng, F. A. Rahim, and M. E. Rusli, “Smart vision: Assistive device for the visually impaired community using online computer vision service,” in 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS), (Singapore), pp. 730–734, 2019.

Z. Li, F. Han, and K. Zheng, “An rgb-d camera-based wearable device for visually impaired people: Enhanced navigation with reduced social stigma,” Electronics, vol. 14, no. 11, p. 2168, 2025.

H. Hakim and A. Fadhil, “Indoor wearable navigation system using 2d slam based on rgb-d camera for visually impaired people,” in Proceedings of First International Conference on Mathematical Modeling and Computational Science (S. Peng, R. Hao, and S. Pal, eds.), vol. 1292 of Advances in Intelligent Systems and Computing, Singapore: Springer, 2021.

Z. Chen, X. Liu, M. Kojima, Q. Huang, and T. Arai, “A wearable navigation device for visually impaired people based on the real-time semantic visual slam system,” Sensors, vol. 21, no. 4, p. 1536, 2021.

F. Song, Z. Li, B. Clark, D. Grooms, and C. Liu, “Camera-based indoor navigation in known environments with orb for people with visual impairment,” in 2020 IEEE Global Humanitarian Technology Conference (GHTC), (Seattle, WA, USA), pp. 1–8, 2020.

S. u. Rahman, S. Ullah, and S. Ullah, “A mobile camera based navigation system for visually impaired people,” in Proceedings of the 7th International Conference on Communications and Broadband Networking, (USA), Association for Computing Machinery, 2019.

G. M. B. Catedrilla, “Mobile-based navigation assistant for visually impaired person with real-time obstacle detection using yolo-based deep learning algorithm,” in 2022 5th Asia Conference on Machine Learning and Computing (ACMLC), (Bangkok, Thailand), pp. 63–67, 2022.

Downloads

Published

2026-03-26

How to Cite

Nanda, P. ., Samal, S. R. ., Bandopadhaya, S. ., Pradhan, D. P. ., Ivanov, A. ., & Poulkov, V. . (2026). AIoT Smart Eyewear with Real-time Object and Audio Recognition for Visually Impaired Users. Journal of Mobile Multimedia, 22(01), 41–62. https://doi.org/10.13052/jmm1550-4646.2212

Issue

2026: Vol 22 Iss 1

Section

Articles

Most read articles by the same author(s)