Integrating the CNN Model with the Web for Indonesian Sign Language (BISINDO) Recognition
DOI:
https://doi.org/10.30871/jaic.v9i3.9345Keywords:
deep learning, indonesian sign language, sign language recognition, webAbstract
Effective communication is challenging for deaf individuals in Indonesia, most of whom use Indonesian Sign Language (BISINDO). Sign Language Recognition (SLR) can bridge this communication gap. While Convolutional Neural Networks (CNNs) show high potential for SLR, their practical accessibility remains limited. This research aims to develop a CNN architecture for recognizing BISINDO alphabet signs from static images (still images) and integrate it into an accessible web platform. Using a static vision-based approach, a CNN model was trained on a public dataset (312 images, 26 classes) following standard pre-processing including data augmentation. The model was subsequently integrated into a web interface using Python and the Gradio library. Results demonstrated strong model performance, with validation accuracy reaching 97.44% and a macro-average F1-score of approximately 97.12%. However, classification challenges were identified for visually similar signs ('M' and 'N'). The resulting integrated web application proved functional, exhibited low prediction latency, and showed cross-platform compatibility. This study successfully demonstrates the development of an accurate DL model for static BISINDO alphabet recognition and its practical implementation via a web platform. This contributes to reducing the accessibility gap in SLR technology. Future research is recommended to utilize larger, more varied datasets and explore dynamic sign recognition.
Downloads
References
[1] World Report on Hearing, 1st ed. Geneva: World Health Organization, 2021.
[2] L. Arisandi and B. Satya, “Sistem Klarifikasi Bahasa Isyarat Indonesia (Bisindo) Dengan Menggunakan Algoritma Convolutional Neural Network,” Jurnal Sistem Cerdas, vol. 5, no. 3, pp. 135–146, Dec. 2022, doi: 10.37396/jsc.v5i3.262.
[3] Kemendikbud, “Tunarungu.” 2023. [Online]. Available: https://kbbi.kemdikbud.go.id/entri/tunarungu
[4] BPS, Hasil Long Form Sensus Penduduk 2020. Badan Pusat Statistik Indonesia, 2023. [Online]. Available: https://www.bps.go.id/id/publication/2023/01/27/ffb5939b4393e5b1146a9b91/hasil-long-form-sensus-penduduk-2020.html
[5] Kemendikbud, “Kamus SIBI,” Kementerian Pendidikan dan Kebudayaan. Dec. 2020. Accessed: Jan. 22, 2024. [Online]. Available: https://pmpk.kemdikbud.go.id/sibi/
[6] Pusbisindo, “Tentang Pusat Bahasa Isyarat Indonesia.” [Online]. Available: https://pusbisindo.org/tentang-kami
[7] O. Kembuan, G. Caren Rorimpandey, and S. Milian Tompunu Tengker, “Convolutional Neural Network (CNN) for Image Classification of Indonesia Sign Language Using Tensorflow,” in 2020 2nd International Conference on Cybernetics and Intelligent
System (ICORIS), Manado, Indonesia: IEEE, Oct. 2020, pp. 1–5. doi: 10.1109/ICORIS50180.2020.9320810.
[8] I. A. Adeyanju, O. O. Bello, and M. A. Adegboye, “Machine learning methods for sign language recognition: A critical review and analysis,” Intelligent Systems with Applications, vol. 12, p. 200056, Nov. 2021, doi: 10.1016/j.iswa.2021.200056.
[9] M. Alaghband, H. R. Maghroor, and I. Garibay, “A survey on sign language literature,” Machine Learning with Applications, vol. 14, p. 100504, Dec. 2023, doi: 10.1016/j.mlwa.2023.100504.
[10] M. Madhiarasan and P. P. Roy, “A Comprehensive Review of Sign Language Recognition: Different Types, Modalities, and Datasets.” arXiv, Apr. 2022. doi: 10.48550/arXiv.2204.03328.
[11] A. Aljabar and S. Suharjito, “BISINDO (Bahasa Isyarat Indonesia) Sign Language Recognition Using CNN and LSTM,” Advances in Science, Technology and Engineering Systems Journal, vol. 5, no. 5, pp. 282–287, 2020, doi: 10.25046/aj050535.
[12] F. Alrowais, S. S. Alotaibi, S. Dhahbi, R. Marzouk, A. Mohamed, and A. Mustafa Hilal, “Sign Language Recognition and Classification Model to Enhance Quality of Disabled People,” Computers, Materials & Continua, vol. 73, no. 2, pp. 3419–3432, 2022, doi: 10.32604/cmc.2022.029438.
[13] N. Adaloglou et al., “A Comprehensive Study on Deep Learning-Based Methods for Sign Language Recognition,” IEEE Transactions on Multimedia, vol. 24, pp. 1750–1762, 2022, doi: 10.1109/TMM.2021.3070438.
[14] S. Dwijayanti, H. -, S. I. Taqiyyah, H. Hikmarika, and B. Y. Suprapto, “Indonesia Sign Language Recognition using Convolutional Neural Network,” International Journal of Advanced Computer Science and Applications, vol. 12, no. 10, 2021, doi: 10.14569/IJACSA.2021.0121046.
[15] A. N. Sihananto, E. M. Safitri, Y. Maulana, F. Fakhruddin, and M. E. Yudistira, “Indonesian Sign Language Image Detection Using Convolutional Neural Network (CNN) Method,” Inspiration: Jurnal Teknologi Informasi dan Komunikasi, vol. 13, no. 1, pp. 13–21, May 2023, doi: 10.35585/inspir.v13i1.37.
[16] S. A. Sanjaya and H. Faustine Ilone, “BISINDO Sign Language Recognition: A Systematic Literature Review of Deep Learning Techniques for Image Processing,” Indonesian Journal of Computer Science, vol. 12, no. 6, Dec. 2023, doi: 10.33022/ijcs.v12i6.3539.
[17] R. Borman, B. Priopradono, and A. Syah, “Klasifikasi Objek Kode Tangan pada Pengenalan Isyarat Alphabet Bahasa Isyarat Indonesia (BISINDO),” SNIA (Seminar Nasional Informatika dan Aplikasinya), vol. 3, 2019, [Online]. Available: https://snia.unjani.ac.id/web/index.php/snia/article/view/87
[18] A. Noer, “Bahasa Isyarat Indonesia (BISINDO) Alphabets.” Kaggle, 2022. [Online]. Available: https://www.kaggle.com/datasets/achmadnoer/alfabet-bisindo
[19] V. R. Joseph, “Optimal Ratio for Data Splitting,” Statistical Analysis, vol. 15, no. 4, pp. 531–538, Aug. 2022, doi: 10.1002/sam.11583.
[20] C.-H. Lin, C. Kaushik, E. L. Dyer, and V. Muthukumar, “The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective,” 2022, arXiv. doi: 10.48550/ARXIV.2210.05021.
[21] A. Abid, A. Abdalla, A. Abid, D. Khan, A. Alfozan, and J. Zou, “Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild,” 2019, arXiv. doi: 10.48550/ARXIV.1906.02569.
[22] S. Daniels, N. Suciati, and C. Fathichah, “Indonesian Sign Language Recognition using YOLO Method,” IOP Conf. Ser.: Mater. Sci. Eng., vol. 1077, no. 1, p. 012029, Feb. 2021, doi: 10.1088/1757-899X/1077/1/012029.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Enisda Libra Kelana, Muhammad Riko Anshori Prasetya, Mambang ., Muhammad Zulfadhilah
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
