Science Journal of Circuits, Systems and Signal Processing

| Peer-Reviewed |

Multiple Sign Language Identification Using Deep Learning Techniques

Received: Jan. 21, 2023    Accepted: Feb. 14, 2023    Published: May 29, 2023
Views:       Downloads:

Share

Abstract

The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.

DOI 10.11648/j.cssp.20231101.11
Published in Science Journal of Circuits, Systems and Signal Processing ( Volume 11, Issue 1, June 2023 )
Page(s) 1-11
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.

Copyright

Copyright © The Author(s), 2024. Published by Science Publishing Group

Keywords

SLI, Capsule Neural Network, Deep Learning, VGG16, LeNet

References
[1] J. Quer and M. Steinbach, "Handling sign language data: The impact of modality," Frontiers in Psychology, 2019.
[2] R. Kushalnagar, "Deafness and hearing loss," Human–Computer Interaction Series, pp. 35-47, 2019.
[3] B. I. II, "Deafness in the Arab world: a general investigation, with applications to Lebanon," scholarship.tricolib.brynmawr.edu, 2018.
[4] C. D. Monteiro, C. M. Mathew, R. Gutierrez-Osuna and F. Shipman, "Detecting and identifying sign languages through visual features," 2016 IEEE International Symposium on Multimedia (ISM), 2016.
[5] A. Sultan, W. Makram, M. Kayed and A. A. ALi, "Sign language identification and recognition: A comparative study," Open Computer Science, pp. 191-210, 2022.
[6] M. Mohandes, Junzhao Liu and M. Deriche, "A survey of image-based Arabic sign language recognition," 2014 IEEE 11th International Multi-Conference on Systems, Signals & Devices (SSD14), 2014.
[7] R. Daroya, D. Peralta and P. Naval, "Alphabet sign language image classification using deep learning," TENCON 2018 - 2018 IEEE Region 10 Conference, 2018.
[8] E. Goceri, "Analysis of capsule networks for Image Classification," Proceedings of the 15th International Conference on Computer Graphics, Visualization, Computer Vision and Image Processing (CGVCVIP 2021), the 7th International Conference on Connected Smart Cities (CSC 2021) and 6th International Conference on Big Data A, 2021.
[9] S. Sabour, N. Frosst and H. Geoffrey E, "Dynamic routing between capsules," Advances in neural information processing systems, 2017.
[10] Y. LeCun, B. Yoshua and H. Geoffrey, "Deep learning," nature, vol. 521, no. 7553, pp. 436-444, 2015.
[11] A. El Zaar, N. Benaya and A. El Allati, "Sign language recognition: High performance deep learning approach applyied to multiple sign languages," E3S Web of Conferences, 2022.
[12] H. T. Nguyen, L. T. Pham, T. T. Mai, T. K. Vo and T. T. Dien, "Letter recognition in hand sign language with VGG-16," Intelligent Systems and Networks, pp. 410-417, 2022.
[13] Z. Alsaadi, E. Alshamani, M. Alrehaili, A. A. Alrashdi, S. Albelwi and A. O. Elfaki, "A real time Arabic Sign Language Alphabets (arsla) recognition model using deep learning architecture," Computers, vol. 11, no. 5, p. 78, 2022.
[14] A.-J. Tanseem N and A.-J. Abu-Jamie, "Classification of Sign-language Using VGG16," 2022.
[15] V. Shreya and Y. Shaik Sohail, "Sign Language Interpreter Using Computer Vision and LeNet-5 Convolutional Neural Network Architecture," International Journal of Innovative Science and Research Technolog, vol. 6, 2021.
[16] M. Bilgin and K. Mutludogan, "American sign language character recognition with capsule networks," 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 2019.
[17] H. Xiao, Y. Yang, K. Yu, J. Tian, X. Cai, U. Muhammad and J. Chen, "Sign language digits and alphabets recognition by Capsule Networks," Journal of Ambient Intelligence and Humanized Computing, vol. 13, no. 4, pp. 2131-2141, 2021.
[18] A. M. B. S. A. K. M. B. and M., "Different techniques of hand segmentation in the real time," IJCAIT, pp. 45-49, 2013.
[19] Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard and L. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Computation, pp. 541-551, 1989.
[20] S. Karen and Z. Andrew, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv: 1409.1556, 2014.
[21] G. B and S. Natarajan, "Hyperparameter optimisation for Capsule Networks," EAI Endorsed Transactions on Cloud Systems, 2019.
[22] J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database," 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009.
[23] K. Suri and R. Gupta, "Continuous sign language recognition from wearable Imus using deep capsule networks and game theory," Computers & Electrical Engineering, vol. 78, pp. 493-503, 2019.
Cite This Article
  • APA Style

    Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali. (2023). Multiple Sign Language Identification Using Deep Learning Techniques. Science Journal of Circuits, Systems and Signal Processing, 11(1), 1-11. https://doi.org/10.11648/j.cssp.20231101.11

    Copy | Download

    ACS Style

    Ahmed Mahmoud Sultan; Waleed Makram Mohamed Zaki; Mohammed Kayed; Abdel Mgeid Amin Ali. Multiple Sign Language Identification Using Deep Learning Techniques. Sci. J. Circuits Syst. Signal Process. 2023, 11(1), 1-11. doi: 10.11648/j.cssp.20231101.11

    Copy | Download

    AMA Style

    Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali. Multiple Sign Language Identification Using Deep Learning Techniques. Sci J Circuits Syst Signal Process. 2023;11(1):1-11. doi: 10.11648/j.cssp.20231101.11

    Copy | Download

  • @article{10.11648/j.cssp.20231101.11,
      author = {Ahmed Mahmoud Sultan and Waleed Makram Mohamed Zaki and Mohammed Kayed and Abdel Mgeid Amin Ali},
      title = {Multiple Sign Language Identification Using Deep Learning Techniques},
      journal = {Science Journal of Circuits, Systems and Signal Processing},
      volume = {11},
      number = {1},
      pages = {1-11},
      doi = {10.11648/j.cssp.20231101.11},
      url = {https://doi.org/10.11648/j.cssp.20231101.11},
      eprint = {https://download.sciencepg.com/pdf/10.11648.j.cssp.20231101.11},
      abstract = {The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.},
     year = {2023}
    }
    

    Copy | Download

  • TY  - JOUR
    T1  - Multiple Sign Language Identification Using Deep Learning Techniques
    AU  - Ahmed Mahmoud Sultan
    AU  - Waleed Makram Mohamed Zaki
    AU  - Mohammed Kayed
    AU  - Abdel Mgeid Amin Ali
    Y1  - 2023/05/29
    PY  - 2023
    N1  - https://doi.org/10.11648/j.cssp.20231101.11
    DO  - 10.11648/j.cssp.20231101.11
    T2  - Science Journal of Circuits, Systems and Signal Processing
    JF  - Science Journal of Circuits, Systems and Signal Processing
    JO  - Science Journal of Circuits, Systems and Signal Processing
    SP  - 1
    EP  - 11
    PB  - Science Publishing Group
    SN  - 2326-9073
    UR  - https://doi.org/10.11648/j.cssp.20231101.11
    AB  - The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.
    VL  - 11
    IS  - 1
    ER  - 

    Copy | Download

Author Information
  • Computer Science Department, Faculty of Computers and Artificial Intelligence, Beni-Suef, Egypt

  • Information System Department, Faculty of Computers and Information Systems, Minia, Egypt

  • Computer Science Department, Faculty of Computers and Artificial Intelligence, Beni-Suef, Egypt

  • Computer Science Department, Faculty of Computers and Information Systems, Minia, Egypt

  • Section