American Journal of Artificial Intelligence

Archive

Submit a Manuscript

Publishing with us to make your research visible to the widest possible audience.

Propose a Special Issue

Building a community of authors and readers to discuss the latest research and develop new ideas.

Research Article |

Signed Language Translation into Afaan Oromo Text Using Deep-Learning Approach

A person who is unable to talk or hear anything can communicate via sign language. For those who have trouble hearing, sign language is a great way to communicate their thoughts and feelings. The vocabulary, grammar, and allied lexicons of sign language are well- defined. This study focuses primarily on Signed Afaan Oromo. The main issue in our society is the detection of Sign Language for the Afaan Oromo language. The construction of static word level, alphabet, and number translations into their equivalent Afaan Oromo text is the main focus of this thesis study. Video frames are used as the system's input, and Afaan Oromo text is used as the system's ultimate output. Data from 90 classes at the alphabet, number, and word level from five special needs instructors have been collected as part of an experiment and literature study to help answer the research objectives. Preprocessing, such as frame extraction, resizing, labeling, and splitting data using Roboflow, as well as the conversion of photos into Yolo model format, was done in order to train our model. Finally, based on the results of our experiment, we can quickly and effectively recognize and classify gestures using data sets of a medium size. The image, webcam, and video file's promising value and forecast results indicate that the yolov5 algorithm has a good chance of successfully detecting the sign in real-time. We trained and tested the model using a signed Afaan Oromo dataset. The YOLOv5s model was successful in obtaining accuracy of 90%, recall of 92.5%, mAP of 93.2% at 0.5 IoU, and a score of 71.5% at 0.5:0.95 IoU, which is suitable for real-time gesture translation.

Signed Language, Deep Learning, Computer Vision, CNN, YOLOv5

APA Style

Negash Tesso, D., Fikadu Dinsa, E., Fikadu Kenani, H. (2023). Signed Language Translation into Afaan Oromo Text Using Deep-Learning Approach. American Journal of Artificial Intelligence, 7(2), 40-51. https://doi.org/10.11648/j.ajai.20230702.12

ACS Style

Negash Tesso, D.; Fikadu Dinsa, E.; Fikadu Kenani, H. Signed Language Translation into Afaan Oromo Text Using Deep-Learning Approach. Am. J. Artif. Intell. 2023, 7(2), 40-51. doi: 10.11648/j.ajai.20230702.12

AMA Style

Negash Tesso D, Fikadu Dinsa E, Fikadu Kenani H. Signed Language Translation into Afaan Oromo Text Using Deep-Learning Approach. Am J Artif Intell. 2023;7(2):40-51. doi: 10.11648/j.ajai.20230702.12

Copyright © 2023 Authors retain the copyright of this article.
This article is an open access article distributed under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. A. Tsegay and K. Raimond, “Offline candidate hand gesture selection and trajectory determination for continuous Ethiopian sign language,” J. Theor. Appl. Inf. Technol., vol. 36, no. 1, pp. 145–153, 2012.
2. Ibrahim Bedane, “The Origin of Afaan Oromo : Mother Language,” Double Blind Peer Rev. Int. Res. J., vol. 15, no. 12, 2015.
3. D. Bijiga, “The Development of Oromo Writing System School of European Culture and Languages,” Http://Kar.Kent.Ac.Uk, pp. 1–295, 2015, [Online]. Available: https://kar.kent.ac.uk/52387/
4. V. Adithya and R. Rajesh, “A Deep Convolutional Neural Network Approach for Static Hand Gesture Recognition,” Procedia Comput. Sci., vol. 171, no. 2019, pp. 2353–2361, 2020, doi: 10.1016/j.procs.2020.04.255.
5. M. Nanda, “You Only Gesture Once (Yougo): American Sign Language Translation Using Yolov3,” no. May, 2020.
6. K. Brady et al., “American Sign Language Recognition and Translation Feasibility Study. | National Technical Reports Library - NTIS,” pp. 3–76, 2018, [Online]. Available: https://ntrl.ntis.gov/NTRL/dashboard/searchResults/titleDetail/AD1098892.xhtml
7. H. D. Patel and A. Saluja, “Sign Language Recognition and Translator Application,” Int. Res. J. Eng. Technol., pp. 652–658, 2021, [Online]. Available: www.irjet.net
8. A. Halder and A. Tayade, “Sign Language to Text and Speech Translation in Real Time Using Convolutional Neural Network,”,” Int. J. Res. Publ. Rev., vol. 8, no. 2, pp. 9–17, 2021.
9. S. Krishnamurthi and M. Indiramma, “Sign Language Translator Using Deep Learning Techniques,” 2021 4th Int. Conf. Electr. Comput. Commun. Technol. ICECCT 2021, 2021, doi: 10.1109/ICECCT52121.2021.9616795.
10. K. Yin, M. Alikhani, A. Moryossef, J. Hochgesang, and Y. Goldberg, “Including Signed Languages in Natural Language Processing (Extended Abstract),” IJCAI Int. Jt. Conf. Artif. Intell., pp. 5369–5373, 2022, doi: 10.24963/ijcai.2022/753.
11. W. Chen, Z. Liqiang, Y. Tianpeng, J. Tao, J. Yijing, and L. Zhihao, “Research on the state detection of the secondary panel of the switchgear based on the YOLOv5 network model,” J. Phys. Conf. Ser., vol. 1994, no. 1, 2021, doi: 10.1088/1742-6596/1994/1/012030.
12. S. S. Sharma et al., “Effective Face Detector Based on YOLOv5 and Superresolution Reconstruction,” Comput. Math. Methods Med., vol. 2021, no. June, pp. 1–9, 2021, doi: 10.1155/2021/7748350.
13. L. Jiang, H. Liu, H. Zhu, and G. Zhang, “Improved YOLO v5 with balanced feature pyramid and attention module for traffic sign detection,” MATEC Web Conf., vol. 355, p. 03023, 2022, doi: 10.1051/matecconf/202235503023.
14. S. M. Ali, “Comparative Analysis of YOLOv3, YOLOv4 and YOLOv5 for Sign Language Detection,” Int. J. Adv. Res. Innov. Ideas Educ., vol. 7, no. 4, pp. 2393–2398, 2021, [Online]. Available: http://ijariie.com/AdminUploadPdf/Comparative_Analysis_of_YOLOv3__YOLOv4_and_YOLOv5_for_Sign_Language_Detection_ijariie15253.pdf
15. D. Thuan, “Evolution of Yolo Algorithm and Yolov5: the State-of-the-Art Object Detection Algorithm,” p. 61, 2021.
16. T. F. Dima and M. E. Ahmed, “Using YOLOv5 Algorithm to Detect and Recognize American Sign Language,” 2021 Int. Conf. Inf. Technol. ICIT 2021 - Proc., no. December, pp. 603–607, 2021, doi: 10.1109/ICIT52682.2021.9491672.
17. Q. Xu, Z. Zhu, H. Ge, Z. Zhang, and X. Zang, “Effective Face Detector Based on YOLOv5 and Superresolution Reconstruction,” Comput. Math. Methods Med., vol. 2021, 2021, doi: 10.1155/2021/7748350.
18. J. H. Kim, N. Kim, Y. W. Park, and C. S. Won, “Object Detection and Classification Based on YOLO-V5 with Improved Maritime Dataset,” J. Mar. Sci. Eng., vol. 10, no. 3, 2022, doi: 10.3390/jmse10030377.
19. W. Wu et al., “Application of local fully Convolutional Neural Network combined with YOLO v5 algorithm in small target detection of remote sensing image,” PLoS One, vol. 16, no. 10 October, pp. 1–15, 2021, doi: 10.1371/journal.pone.0259283.
20. Z. Q. Zhao, P. Zheng, S. T. Xu, and X. Wu, “Object Detection with Deep Learning: A Review,” IEEE Trans. Neural Networks Learn. Syst., vol. 30, no. 11, pp. 3212–3232, 2019, doi: 10.1109/TNNLS.2018.2876865.
21. W. Jia et al., “Real-time automatic helmet detection of motorcyclists in urban traffic using improved YOLOv5 detector,” IET Image Process., vol. 15, no. 14, pp. 3623–3637, 2021, doi: 10.1049/ipr2.12295.
22. T. Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal Loss for Dense Object Detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 2, pp. 318–327, 2020, doi: 10.1109/TPAMI.2018.2858826.
23. A. Ananthanarayan, “Hand Detection and Body Language Recognition Using YOLO,” no. May, 2021, [Online]. Available: https://hdl.handle.net/1969.1/188381
24. M. Mulugeta, “Development of an Amharic Speech to Ethiopian Sign Language Translation System.,” Bahirdar universty, Ethiop., 2016.
25. D. Solanki, “OBJECT DETECTION AND CLASSIFICATION THROUGH DEEP LEARNING APPROACHES,” vol. 5, no. 9, pp. 230–235, 2018.
26. S. L. Kuna, “Real Time Object Detection and Tracking using Deep Learning,” Int. J. Res. Appl. Sci. Eng. Technol., vol. 8, no. 6, pp. 58–64, 2020, doi: 10.22214/ijraset.2020.6010.
27. E. Martinez-Martin and F. Morillas-Espejo, “Deep Learning Techniques for Spanish Sign Language Interpretation,” Comput. Intell. Neurosci., vol. 2021, no. i, 2021, doi: 10.1155/2021/5532580.
28. M. Rivera-Acosta, J. M. Ruiz-Varela, S. Ortega-Cisneros, J. Rivera, R. Parra-Michel, and P. Mejia-Alvarez, “Spelling correction real-time american sign language alphabet translation system based on yolo network and LSTM,” Electron., vol. 10, no. 9, 2021, doi: 10.3390/electronics10091035.
29. S. Daniels, N. Suciati, and C. Fathichah, “Indonesian Sign Language Recognition using YOLO Method,” IOP Conf. Ser. Mater. Sci. Eng., vol. 1077, no. 1, p. 012029, 2021, doi: 10.1088/1757-899x/1077/1/012029.
30. I. Cardoza, J. P. García-Vázquez, A. Díaz-Ramírez, and V. Quintero-Rosas, “Convolutional Neural Networks Hyperparameter Tunning for Classifying Firearms on Images,” Appl. Artif. Intell., vol. 36, no. 1, pp. 1–23, 2022, doi: 10.1080/08839514.2022.2058165.
31. K. Guo, C. He, M. Yang, and S. Wang, “A pavement distresses identification method optimized for YOLOv5s,” Sci. Rep., vol. 12, no. 1, pp. 1–15, 2022, doi: 10.1038/s41598-022-07527-3.
32. D. K. Singh, “3D-CNN based Dynamic Gesture Recognition for Indian Sign Language Modeling,” Procedia CIRP, vol. 189, pp. 76–83, 2021, doi: 10.1016/j.procs.2021.05.071.
33. J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement,” 2018, [Online]. Available: http://arxiv.org/abs/1804.02767