Indonesian Sign Language Application Using Mediapipe and Gated Recurrent Unit in Real Time
DOI:
https://doi.org/10.30871/jaic.v10i2.12150Keywords:
BISINDO, Deep Learning, Gated Recurrent Unit, MediaPipe, Real Time RecognitionAbstract
Indonesian Sign Language (BISINDO) is a natural communication tool for the deaf community. However, the communication gap between signers and the general public remains a challenge due to the dynamic nature of sign language. This study proposes a real-time recognition system using the Gated Recurrent Unit (GRU) method. The system utilizes MediaPipe Holistic to extract 1,662 spatial keypoints, which are then processed as temporal sequences of 60 frames. The dataset comprises 300 video samples of ten dynamic BISINDO gestures ('apa', 'bapak', 'dimana', 'halo', 'hari', 'ibu', 'kabar', 'siapa', ‘kenapa’, ‘rumah’) recorded from multi dependent user under consistent indoor lighting conditions. The proposed model architecture consists of two GRU layers with Batch Normalization and Dropout to optimize performance with a total of 415,946 parameters. Results show that the model successfully achieves 86,67% accuracy and efficient inference speeds, making it suitable for real-time application on standard computing devices. This research serves as a proof-of-concept for assistive communication technology.
Downloads
References
[1] L. Arisandi and B. Satya, “Sistem Klarifikasi Bahasa Isyarat Indonesia (Bisindo) Dengan Menggunakan Algoritma Convolutional Neural Network,” J. Sist. Cerdas, vol. 5, no. 3, pp. 135–146, Dec. 2022, doi: 10.37396/jsc.v5i3.262.
[2] C. Artamma and M. Rahardi, “L2IC and MobileViT-XXS for BISINDO Alphabet Recognition,” vol. 9, no. 6.
[3] M. Kharis, M. Andika, and H. A. D. Rani, “Aplikasi Augmented Reality Interaktif Tuna Rungu Berbasis Android pada Sekolah Luar Biasa ABCD Muhammadiyah Palu,” 2024.
[4] F. M. Dewanto, A. T. J. Harjanta, N. Q. Nada, and B. Agus, “Pengenalan Gestur Bahasa Isyarat Indonesia dengan Mediapipe Keypoints,” vol. 6, no. 2, 2024.
[5] R. I. Borman and B. Priyopradono, “Implementasi Penerjemah Bahasa Isyarat Pada Bahasa Isyarat Indonesia (BISINDO) Dengan Metode Principal Component Analysis (PCA),” J. Inform. J. Pengemb. IT, vol. 3, no. 1, pp. 103–108, Jan. 2018, doi: 10.30591/jpit.v3i1.631.
[6] R. A. Saputra, F. M. Setiawan, E. Ryansyah, and C. Rozikin, “Pendeteksi Bahasa Isyarat Menggunakan TensorFlow dengan Metode Convolutional Neural Network,” J. Inform. Dan Rekayasa KomputerJAKAKOM, vol. 5, no. 2.
[7] M. Feliciano, W. Gunawan, and N. E. Jahja, “American Sign Language (Asl) Recognition Using The Cnn Method,” . Vol., vol. 7, no. 2.
[8] Y. Brianorman and R. Munir, “Perbandingan Pre-Trained CNN: Klasifikasi Pengenalan Bahasa Isyarat Huruf Hijaiyah,” J Sist. Info Bisnis, vol. 13, no. 1, pp. 52–59, Jul. 2023, doi: 10.21456/vol13iss1pp52-59.
[9] E. Tikasni, E. Utami, and D. Ariatmanto, “Analisis Akurasi Object Detection Menggunakan Tensorflow Untuk Pengenalan Bahasa Isyarat Tangan Menggunakan Metode SSD,” J. FASILKOM, vol. 14, no. 2, pp. 385–393, Aug. 2024, doi: 10.37859/jf.v14i2.7512.
[10] A. Widya Agata, W. S J Saputra, and C. Aji Putra, “Pengenalan Bahasa Isyarat Indonesia (BISINDO) Menggunakan Algoritma Sscale Invariant Feature Transform (SIFT) Dan Convolutional Neural Network (CNN),” JATI J. Mhs. Tek. Inform., vol. 8, no. 1, pp. 1054–1061, Mar. 2024, doi: 10.36040/jati.v8i1.8917.
[11] F. A. Febrianti, M. A. Rahman, N. Alani, R. Nuriyanti, and I. P. Sari, “Pemanfaatan Website Interaktif Berbasis Bahasa Isyarat Indonesia (BISINDO) sebagai Media Pembelajaran Inklusif bagi Anak Sekolah Dasar,” 2025.
[12] C. Lugaresi et al., “MediaPipe: A Framework for Perceiving and Processing Reality”.
[13] D. P. Kartaputra, H. Gunawan, and A. E. Lestari, “Deteksi Alfabet BISINDO Menggunakan Mediapipe Holistic Secara Real-Time,” J. Teknol. Inf. Dan Komun., vol. 12, no. 1, pp. 46–52, Jun. 2023, doi: 10.58761/jurtikstmikbandung.v12i1.4652.
[14] S. Nur Budiman, S. Lestanti, S. Marselius Evvandri, and R. Kartika Putri, “Pengenalan Gestur Gerakan Jari Untuk Mengontrol Volume Di Komputer Menggunakan Library OpenCV Dan Mediapipe,” Antivirus J. Ilm. Tek. Inform., vol. 16, no. 2, pp. 223–232, Nov. 2022, doi: 10.35457/antivirus.v16i2.2508.
[15] J. Bora, S. Dehingia, A. Boruah, A. A. Chetia, and D. Gogoi, “Real-time Assamese Sign Language Recognition using MediaPipe and Deep Learning,” Procedia Comput. Sci., vol. 218, pp. 1384–1393, 2023, doi: 10.1016/j.procs.2023.01.117.
[16] P. Kurnia Sari, G. Qorik Oktagalu Pratamasunu, and F. Nur Fajri, “Deteksi Tangan Otomatis Pada Video Percakapan Bahasa Isyarat Indonesia Menggunakan Metode Deep Gated Recurrent Unit (GRU),” J. Komput. Terap., vol. 8, no. 1, pp. 186–193, Jun. 2022, doi: 10.35143/jkt.v8i1.4901.
[17] M. A. Syifa and D. R. S. Saputro, “Stance Detection Dengan Algoritme Gated Recurrent Unit (GRU),” 2023.
[18] I. Sulistiyowati, H. Maulana Ichsan, and I. Anshory, “Konveyor Penyortir Objek Dengan Deteksi Warna Menggunakan Kamera ESP-32 Berbasis OpenCV Python,” Pros. TAU SNARS-TEK Semin. Nas. Rekayasa Dan Teknol., vol. 4, no. 1, pp. 35–41, Aug. 2024, doi: 10.47970/snarstek.v2i1.711.
[19] R. D. Martinez Marenco et al., Automated segmentation of breast cancer in ultrasound images using self-learning deep learning models. Ediciones Universidad Simón Bolívar, 2024. doi: 10.17081/r.book.2025.01.16118.
[20] I. P. Y. Agus Ariwanta, K. Y. Ernanda Aryanto, and I. G. A. Gunadi, “Suricata Accuracy Optimization Based On Live Analysis Using One-Class Support Vector Machine Method And Streamlit Framework,” J. Tek. Inform. Jutif, vol. 5, no. 2, pp. 415–427, Apr. 2024, doi: 10.52436/1.jutif.2024.5.2.1822.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Adi Triswantoro, Febrian Murti Dewanto, Aris Tri Jaka Harjanta
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
