A Moroccan Sign Language Recognition Algorithm Using a Convolution Neural Network

Authors

  • Nourdine Herbaz Laboratory of Electronics, Energy, Automation & Information Processing(LEEA&TI), Faculty of Sciences and Techniques Mohammedia, Hassan II University of Casablanca, 28830, Morocco
  • Hassan El Idrissi Laboratory of Electronics, Energy, Automation & Information Processing(LEEA&TI), Faculty of Sciences and Techniques Mohammedia, Hassan II University of Casablanca, 28830, Morocco
  • Abdelmajid Badri Laboratory of Electronics, Energy, Automation & Information Processing(LEEA&TI), Faculty of Sciences and Techniques Mohammedia, Hassan II University of Casablanca, 28830, Morocco

DOI:

https://doi.org/10.13052/jicts2245-800X.1033

Keywords:

Sign language, convolutional neural networks, deaf-mutes people, image processing, real time

Abstract

Gesture recognition is an open phenomenon in computer vision, and one of the topics of current interest. Gesture recognition has many applications in the interpretation of sign language in deaf-mutes, one is in human-computer interaction, and the other is in immersive game technology.

For this purpose, we have developed a model of image processing recognition of gestures, based on Artificial Neural Networks, starting from data collection, identification, tracking and classification of gestures, to the display of the obtained results. We propose an approach to contribute to the translation of sign language into voice/text format.

In this paper, we present a Moroccan sign language recognition system using a Convolutional Neural Network (CNN). This system includes an important data set of more than 20 files. Each file contains 1,000 static images of each signal from several different angles that we collected with the camera. Different Sign Language models were evaluated and compared with the proposed CNN model. The proposed system achieved 99.33% and achieved the best performance with an accuracy rate of 98.7%.

Downloads

Download data is not yet available.

Author Biographies

Nourdine Herbaz, Laboratory of Electronics, Energy, Automation & Information Processing(LEEA&TI), Faculty of Sciences and Techniques Mohammedia, Hassan II University of Casablanca, 28830, Morocco

Nourdine Herbaz is a Ph.D. student in Electronics, Energy, Automation & Information Processing (LEEA&TI) laboratory, Faculty of Sciences and Techniques of Mohammedia, Hassan II University of Casablanca ,Morocco. He did his bachelor’s degree in Biomedical Instrumentation and Maintenance from Hassan I University in 2016 from Morocco, and Masters’ degree in Embedded systems and Mobile from Tunisia. His research interest is Artificial Intelligence, Neural Network, Embedded systems…. Currently, the project on which he is working is focused on applications of artificial intelligence for the interpretation of sign language in deaf mutes.

Hassan El Idrissi, Laboratory of Electronics, Energy, Automation & Information Processing(LEEA&TI), Faculty of Sciences and Techniques Mohammedia, Hassan II University of Casablanca, 28830, Morocco

Hassan El Idrissi is a full professor since 1994 in the electrical engineering department at the Faculty of Sciences and Techniques of Mohammedia in Hassan II University Casablanca – Morocco, where he teaches courses on the physics of semiconductors, sensors, electronics and graphic programming dedicated to instrumentation, automation, and supervision. His thesis defended in 1993 at the Institute of Electronics and Microelectronics of the North, of the University of Science and Technology of Lille in France, focused on field effect transistors with insulated gate. He has supervised theses in the field of semiconductors and magnetic pulse generators. He has participated in several national and international congresses and conferences. His current research focuses on artificial intelligence and its societal applications, more particularly the electronic coding of sign language in the deaf mute by camera or smart glove.

Abdelmajid Badri, Laboratory of Electronics, Energy, Automation & Information Processing(LEEA&TI), Faculty of Sciences and Techniques Mohammedia, Hassan II University of Casablanca, 28830, Morocco

Abdelmajid Badri is a holder of a doctorate in Electronics and Image Processing in 1992 at the University of Poitiers–France. In 1996, he obtained the diploma of the authorization to Manage Researches (Habilitation à Diriger des Recherches: HDR) to the University of Poitiers– France, on the image processing. Qualifed by the CNU-France in 61th section (informatics Engineering, Automatic and Signal processing. He is an University Professor (PES-C) at the University Hassan II of Casablanca – Morocco (FSTM) where he teaches the electronics, the signal processing, image processing and telecommunication (Department of Electric Engineering). He is a member of the laboratory EEA&TI (Electronics, Electrotechnics, Automatic and information Processing) which he managed since 1996. The research works of A. Badri concerns the communication and Information Technology (Electronics Systems, Signal/Image Processing and Telecommunication). He managed several doctoral theses. He is a co-author of several national and international publications. He is responsible for several research projects financed by the ministry or by the CNRST or by the industrialists. He was member of several committees of programs of international conferences, reviewer of several revues and chairman of several international congresses in the same domain. He is a member and coresponsible in several scientific associations in touch with his domain of research. He is an expert CNRST and Ministry. He was responsible for several academic structures (Director of ESTC, Director of ENSAMC an interim, Vice Dean FSTM, Head of the Electric Engineering Department).

References

S.M. PASCUA; P.L.C. ESPINA; R.P.L. TALAG; L.N. VILLEGAS; L. AQUINO DE GUZMAN. A Filipino Sign Language Thesaurus Management System Using Ren-py. IFLA WLIC 2017 – Wrocław, Poland – Libraries. Solidarity. Society., 2017.

N. El-Bendary; H. Zawbaa; M. Daoud; A.E. Hassanien; K. Nakamatsu. International Journal of Computer Information Systems and Industrial Management Applications, 590–595, 2010.

F. Zhang. Human-Computer Interactive Gesture Feature Capture and Recognition in Virtual Reality. Ergonomics in Design, 29(2):19–25, 2021.

P. Sharma and A.R. Shyam. Depth data and fusion of feature descriptors for static gesture recognition. IET Image Processing, 14(5): 909–920, 2020.

Q. Zheng; M. Yang; X. Tian; N. Jiang and D. Wang. A Full Stage Data Augmentation Method in Deep Convolutional Neural Network for Natural Image Classification. Discret. Dyn. Nat. Soc, 1(11), 2020.

A.R. Asif; A. Waris; S.O. Gilani; M. Jamil; H. Ashraf; M. Shafique; I.K. Niazi. Performance Evaluation of Convolutional Neural Network for Hand Gesture Recognition Using EMG. Sensors, 20(6), 2020.

A. A. I. Sidig; H. Luqman; S. Mahmoud; M. Mohandes. KArSL: Arabic Sign Language Database ACM Transactions on Asian and Low-Resource Language Information Processing, 20(1), 1–-19, 2021.

R. Nair; K.A. Dileep; Ashu; S. Yadav; B. Sourabh. Hand Gesture Recognition system for physically challenged people using IoT. 6th International Conference on Advanced Computing & Communication Systems (ICACCS), 671–675, 2020.

A. Sharma; A. Mittal; S. Singh; V. Awatramani. Hand Gesture Recognition using Image Processing and Feature Extraction Techniques. Procedia Comput. Sci, 173:181–-190, 2020.

J.P. Sahoo; A.J. Prakash; P. Pławiak; S. Samantray. Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network. Sensors, 22(3), 706, 2022.

L. Fang; N. Liang; W. Kang; Z. Wang; D.D. Feng. Real-time hand posture recognition using hand geometric features and Fisher Vector. Signal Processing: Image Communication, 82, p:115729, 2019.

Y.S. Tan; K.M. Lim; C.P. Lee. Hand gesture recognition via enhanced densely connected convolutional neural network. Expert Syst. Appl., 175, p:114797, 2021.

C. Arun; R. Gopikakumari. U Optimisation of both classifier and fusionbased feature set for static American sign language recognition. IET Image Process., 14(10):2101–2109, 2020.

X. Tang; Z. Yan; J. Peng; B. Hao; H. Wang; J. Li. Selective spatiotemporal features learning for dynamic gesture recognition. Expert Syst. Appl., 169, p:114499, 2021.

V. Jain; A. Jain; A. Chauhan; S.S. Kotla; A. Gautam. American Sign Language recognition using Support Vector Machine and Convolutional Neural Network. International Journal of Information Technology..13(3):1193–-1200, 2021.

Downloads

Published

2022-10-20

How to Cite

Herbaz, N. ., El Idrissi, H. ., & Badri, A. . (2022). A Moroccan Sign Language Recognition Algorithm Using a Convolution Neural Network. Journal of ICT Standardization, 10(03), 411–426. https://doi.org/10.13052/jicts2245-800X.1033

Issue

2022: Vol 10 Iss 3

Section

Intelligent Systems for Smart Applications