Analysis of Deep Learning Accuracy for BISINDO Sign Language Translation

Rizal Sengkey; Brave A. Sugiarso; Dirko G. S. Ruindungan

doi:10.33559/eoj.v8i10.3694

Analysis of Deep Learning Accuracy for BISINDO Sign Language Translation

Rizal Sengkey, Brave A. Sugiarso, Dirko G. S. Ruindungan

Sari

Communication between the deaf and hearing people is still a challenge, especially due to the lack of understanding of the sign language they often used. To overcome this, the author developed an Indonesian Sign Language (BISINDO) translation and recognition application by utilizing the YOLOv4 deep learning model. This study aims to analyze the accuracy of the model in detecting and translating sign language in real-time. The research stage begins with creating a model. The model will be trained using the BISINDO dataset which can be accessed through an open-source dataset provider site for computer vision, and the performance evaluation will be carried out through several metrics such as Intersection over Union (IoU), Precision, Recall, and Mean Average Precision (mAP). The expected result is a model that is able to recognize and classify sign language with high accuracy based on the predetermined metrics. This research is useful in providing inclusive technological solutions to facilitate communication between deaf and hearing people, while increasing accessibility in the use of sign language.

Key words — Accuracy; BISINDO; Deep learning; Sign language.

Teks Lengkap:

PDF

Referensi

E. Tikasni, E. Utami, and D. Ariatmanto, “Analisis Akurasi Object Detection Menggunakan Tensorflow Untuk Pengenalan Bahasa Isyarat Tangan Menggunakan Metode SSD,” JURNAL FASILKOM, vol. 14, no. 2, pp. 385–393, Aug. 2024, doi: 10.37859/jf.v14i2.7512.

A. Nugroho, R. Setiawan, A. Harris, and Beny, “Deteksi Bahasa Isyarat Bisindo Menggunakan Metode Machine Learning,” Jurnal PROCESSOR, vol. 18, no. 2, pp. 152–158, Nov. 2023, doi: 10.33998/processor.2023.18.2.1380.

S. N. Budiman, S. Lestanti, H. Yuana, and B. N. Awwalin, “SIBI (Sistem Bahasa Isyarat Indonesia) berbasis Machine Learning dan Computer Vision untuk Membantu Komunikasi Tuna Rungu dan Tuna Wicara,” Jurnal Teknologi dan Manajemen Informatika, vol. 9, no. 2, pp. 119–128, Dec. 2023, doi: 10.26905/jtmi.v9i2.10993.

J. Holdsworth and M. Scapicchio, “What is deep learning?,” IBM. Accessed: Feb. 03, 2025. [Online]. Available: https://www.ibm.com/think/topics/deep-learning

N. C. Camgoz, O. Koller, S. Hadfield, and R. Bowden, “Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle: IEEE, Jun. 2020, pp. 10020–10030. doi: 10.1109/CVPR42600.2020.01004.

O. Koller, N. C. Camgoz, H. Ney, and R. Bowden, “Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos,” IEEE Trans Pattern Anal Mach Intell, vol. 42, no. 9, pp. 2306–2320, Sep. 2020, doi: 10.1109/TPAMI.2019.2911077.

“What are convolutional neural networks?,” IBM. Accessed: Feb. 03, 2025. [Online]. Available: https://www.ibm.com/think/topics/convolutional-neural-networks

A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, “YOLOv4: Optimal Speed and Accuracy of Object Detection,” Apr. 2020.

S. Thakar, S. Shah, B. Shah, and A. V Nimkar, “Sign Language to Text Conversion in Real Time using Transfer Learning,” in 2022 IEEE 3rd Global Conference for Advancement in Technology (GCAT), Bangalore: IEEE, Oct. 2022, pp. 1–5. doi: 10.1109/GCAT55367.2022.9971953.

M. R. Ningsih, Y. J. Nurriski, F. A. Z. Sanjani, M. F. Al Hakim, J. Unjung, and M. A. Muslim, “Sign Language Detection System Using YOLOv5 Algorithm to Promote Communication Equality People with Disabilities,” Scientific Journal of Informatics, vol. 11, no. 2, pp. 549–558, Jun. 2024, doi: https://doi.org/10.15294/sji.v11i2.6007.

H. Alzarooni, H. Al Ali, S. Fadaaq, and T. Bonny, “AI-Based Sign Language Interpreter (GestPret),” in Proceedings of the 2023 7th International Conference on Advances in Artificial Intelligence, New York, NY, USA: ACM, Oct. 2023, pp. 79–85. doi: 10.1145/3633598.3633612.

S. Ennaama, H. Silkan, A. Bentajer, and A. Tahiri, “Enhanced Real-Time Object Detection using YOLOv7 and MobileNetv3,” Engineering, Technology & Applied Science Research, vol. 15, no. 1, pp. 19181–19187, Feb. 2025, doi: 10.48084/etasr.8777.

R. Padilla, W. L. Passos, T. L. B. Dias, S. L. Netto, and E. A. B. da Silva, “A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit,” Electronics (Basel), vol. 10, no. 3, pp. 279–306, Jan. 2021, doi: 10.3390/electronics10030279.

L. H. He, Y. Z. Zhou, L. Liu, W. Cao, and J. H. Ma, “Research on object detection and recognition in remote sensing images based on YOLOv11,” Sci Rep, vol. 15, no. 1, p. 14032, Apr. 2025, doi: 10.1038/s41598-025-96314-x.

W. Ying et al., “A Survey on Data-Centric AI: Tabular Learning from Reinforcement Learning and Generative AI Perspective,” Feb. 2025.

DOI: https://doi.org/10.33559/eoj.v8i10.3694