Image Processing and Object Detection in the Indonesian Sign System (SIBI) for Hearing-Impaired Communication
DOI:
https://doi.org/10.30871/jaic.v10i1.11395Keywords:
Communcation, Indonesian Sign Language (SIBI), Computer Vision, Real-Time Detection, Web-Based ApplicationAbstract
Communication is a fundamental human need, yet individuals with hearing impairments continue to face barriers due to limited access to sign language translation technologies. In Indonesia, the adoption of such technologies remains low, particularly in regions such as Sorong, Southwest Papua, creating a communication gap between the Deaf community and the general public. This study develops a web-based detection system for 36 classes of the Indonesian Sign System (SIBI) using the YOLOv5 algorithm. The dataset consists of 5,682 images of SIBI hand poses with variations in lighting and background, divided into 4,970 training images (87%), 376 validation images (7%), and 335 test images (6%). All data were processed through labeling, preprocessing, augmentation, balancing, and model training. The training was conducted for 150 epochs, and the evaluation results show that YOLOv5 is capable of detecting SIBI signs with significant accuracy. Performance evaluation using a confusion matrix achieved a detection accuracy of 95%, supported by stable precision and recall values and real-time inference performance on common web browsers. Usability testing with 20 respondents indicated satisfaction levels above 72.8%, demonstrating that the system is practical and easy to use. This research presents a validated real-time, web-based SIBI detection system that supports inclusive computer vision applications and enhances accessibility in public services such as education, healthcare, and administrative environments.
Downloads
References
[1] S. M. Fitriyani, Lathifah Qurrotu Ainii, Raudhatul Jannah, “Analysis of Sign Language Skills in Improving Communication and Learning for Deaf Children,” vol. 5, no. 1, pp. 167–186, 2021.
[2] B. O. Olusanya, A. C. Davis, and H. J. Hoffman, “Hearing loss grades and the International classification of functioning, disability and health.,” Bull. World Health Organ., vol. 97, no. 10, pp. 725–728, Oct. 2019, doi: 10.2471/BLT.19.230367.
[3] Survey kesehatan indonesia (Ski), “Survei Kesehatan Indonesia 2023 (SKI),” Kemenkes, p. 235, 2023.
[4] R. Zaskia, R. Khairunnisa, and L. S. Fernanda, “Hubungan Gangguan Pendengaran Dengan Kemampuan Berbahasa Pada Anak – Anak,” J. Skripta, vol. 11, no. 1, pp. 54–59, 2025, doi: 10.31316/skripta.v11i1.7185.
[5] Heriyanti, Jeane Susan Tapada, Kris Uluelang, Putri Jalila Umasugi, and Isnaeni Wahab, “Teacher’S Strategies in Teaching English To Deaf Students in a Special School Sorong,” JLE J. Lit. English Educ. Study Progr., vol. 4, no. 02, pp. 91–101, 2023, doi: 10.47435/jle.v4i02.2332.
[6] Adam Nurmansyah, Nanda Rizqia Rhamadhani, Sabrina Alfarissy Nur Hakim, Sri Azhari Agustin, and Siti Hamidah, “Permasalahan Komunikasi Yang Kerap Terjadi Pada Penyandang Disabilitas,” J. Pendidikan, Bhs. dan Budaya, vol. 2, no. 2, pp. 200–210, 2023, doi: 10.55606/jpbb.v2i2.1515.
[7] H. Halim and L. Lina, “Aplikasi Pengidentifikasi Bahasa Isyarat Berdasarkan Gerak Tubuh Secara Real Time Menggunakan Yolo,” Simtek J. Sist. Inf. dan Tek. Komput., vol. 8, no. 2, pp. 300–304, 2023, doi: 10.51876/simtek.v8i2.215.
[8] T. Pramanik, P. G. Burade, and S. Sharma, “Analysis of real-time multi-surveillance detection model using YOLO v5,” Indones. J. Electr. Eng. Comput. Sci., vol. 38, no. 3, p. 1634, 2025, doi: 10.11591/ijeecs.v38.i3.pp1634-1641.
[9] A. M. Hidayahtullah, “Sistem Deteksi Simbol Pada SIBI (Sistem Isyarat Bahasa Indonesia) Secara Realtime menggunakan Mobilenet-SSD,” dinamika, vol. 33, no. 1, pp. 1–12, 2022.
[10] M. Ugale, O. R. A. Shinde, K. Desle, and S. Yadav, A Review on Sign Language Recognition Using CNN. Atlantis Press International BV, 2023. doi: 10.2991/978-94-6463-136-4_23.
[11] L. Sandy, A. Putra, R. R. Yacoub, and E. Kusumawardhani, “SIBI Introduction Using YOLOv8 as Android-Based Learning Media,” vol. 3, no. 1, pp. 24–33, 2025, doi: 10.26418/telectrical.v3i1.93519.
[12] D. Luthfy, C. Setianingshi, and M. W. Paryasto, “Indonesian Sign Language Classification Using You Only Look Once,” eProceedings Eng., vol. 10, no. 1, pp. 454–459, 2023.
[13] W. Saputro and D. B. Sumantri, “Klasifikasi Citra Dalam Gerak Tangan Bahasa Isyarat Sibi Menggunakan Algoritma K-NN,” INTECOMS J. Inf. Technol. Comput. Sci., vol. 5, no. 2, pp. 180–188, 2022, doi: 10.31539/intecoms.v5i2.4446.
[14] B. K. Pratama, Sri Lestanti, and Yusniarsi Primasari, “Implementasi Algoritma You Only Look Once (YOLO) untuk Mendeteksi Bahasa Isyarat SIBI,” ProTekInfo(Pengembangan Ris. dan Obs. Tek. Inform., vol. 11, no. 2, pp. 7–14, 2024, doi: 10.30656/protekinfo.v11i2.9105.
[15] R. Hesananda, I. A. Noviani, and M. Zulfariansyah, “Implementasi YOLOv5 untuk Deteksi Objek Mesin EDC: Evaluasi dan Analisis,” BIOS J. Teknol. Inf. dan Rekayasa Komput., vol. 5, no. 2, pp. 104–110, 2024, doi: 10.37148/bios.v5i2.127.
[16] Salsabilla Azahra Putri, Murinto, and Sunardi, “Implementasi Algoritma YOLOv5 untuk Otomatisasi Iklan Layananan Publik tentang Larangan Merokok,” J. Inform. Polinema, vol. 11, no. 2, pp. 195–202, 2025, [Online]. Available: https://jurnal.polinema.ac.id/index.php/jip/article/view/6343
[17] M. Vicko, P. Ardiansyah, J. Sahertian, and R. Heri, “Penerapan Algoritma YOLO dalam Sistem Klasifikasi Kendaraan,” vol. 9, pp. 1151–1157, 2025.
[18] P. Explained and K. Beck, Praise for Extreme Programming Explained , Second Edition.
[19] J. Redmon, “YOLOv3: An Incremental Improvement”.
[20] R. Rudianto, “Penerapan Metode Extreme Programming Dalam Pembangunan Aplikasi Sistem Penunjang Keputusan,” J. Sist. Inf. dan Bisnis Cerdas, vol. 16, no. 1, pp. 21–30, 2023, doi: 10.33005/sibc.v16i1.6.
[21] R. Dwiyanto, D. W. Widodo, and P. Kasih, “Implementasi Metode You Only Look Once ( YOLOv5 ) Untuk Klasifikasi Kendaraan Pada CCTV Kabupaten Tulungagung,” Semin. Nas. Inov. Teknol., vol. 1, no. 1, pp. 102–104, 2022.
[22] A. Pambudi and W. Apriandari, “An Extreme Programming Approach for Instructor Performance Evaluation System Development,” vol. 8106, pp. 126–135, 2023.
[23] A. E. Kalambia, G. Kevin, and P. H. Saputro, “Model Deep Learning YOLOv5 untuk Identifikasi Cuaca: Cloudy, Rain, Shine, dan Sunrise,” Inform. J. Ilmu Komput., vol. 20, no. 3, pp. 125–132, 2024, doi: 10.52958/iftk.v20i3.11089.
[24] M. H. Ashar and D. Suarna, “KLIK: Kajian Ilmiah Informatika dan Komputer Implementasi Algoritma YOLOv5 dalam Mendeteksi Penggunaan Masker Pada Kantor Biro Umum Gubernur Sulawesi Barat,” Media Online, vol. 3, no. 3, pp. 298–302, 2022, [Online]. Available: https://djournals.com/klik
[25] R. Hesananda, “Implementasi Model Yolo V5 Untuk Deteksi Korek Api Dalam Keamanan Penerbangan,” J. Inform. dan Tek. Elektro Terap., vol. 13, no. 1, 2025, doi: 10.23960/jitet.v13i1.5553.
[26] A. Jailani and M. A. Yaqin, “Pengujian Aplikasi Sistem Informasi Akademik menggunakan Metode Blackbox dengan Teknik Boundary Value Analysis,” J. Autom. Comput. Inf. Syst., vol. 4, no. 2, pp. 60–66, 2024, doi: 10.47134/jacis.v4i2.78.
[27] Y. D. Wijaya and M. W. Astuti, “Pengujian Blackbox Sistem Informasi Penilaian Kinerja Karyawan Pt Inka (Persero) Berbasis Equivalence Partitions,” J. Digit. Teknol. Inf., vol. 4, no. 1, p. 22, 2021, doi: 10.32502/digital.v4i1.3163.
[28] A. A. Santika, T. H. Saragih, and M. Muliadi, “Penerapan Skala Likert pada Klasifikasi Tingkat Kepuasan Pelanggan Agen Brilink Menggunakan Random Forest,” J. Sist. dan Teknol. Inf., vol. 11, no. 3, p. 405, 2023, doi: 10.26418/justin.v11i3.62086.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Dewi Astria Faroek, Muhammad Yusuf, Haris Haris
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
