L2IC and MobileViT-XXS for BISINDO Alphabet Recognition
DOI:
https://doi.org/10.30871/jaic.v9i6.11575Keywords:
Indonesian Sign Language (BISINDO), Landmark-to-Image Conversion (L2IC), Deep LearningAbstract
This study proposes a Landmark-to-Image Conversion (L2IC) approach integrated with the MobileViT-XXS architecture for Indonesian Sign Language (BISINDO) alphabet recognition. The method converts 42 hand keypoints, extracted using MediaPipe Hands into normalized 224×224 grayscale images to capture spatial hand patterns more effectively. These L2IC representations are then used as input to the MobileViT-XXS model, trained for 30 epochs with a learning rate of 0.001. Experimental results show that the model achieves an accuracy and Macro F1-Score of 97.98%, outperforming baseline approaches using raw RGB images and MLP-based classification on numerical keypoints. While the model demonstrates strong performance in controlled offline experiments, further evaluation is required to assess its robustness under real-world dynamic BISINDO usage and deployment on resource-limited devices. These findings indicate that the L2IC representation effectively captures essential spatial information, contributing to high recognition accuracy in static BISINDO hand gesture classification.
Downloads
References
[1] “Potret Penyandang Disabilitas di Indonesia: Hasil Long Form SP2020 - Badan Pusat Statistik Indonesia.” Accessed: Oct. 15, 2025.[Online].Available: https://www.bps.go.id/id/publication/2024/12/20/43880dc0f8be5ab92199f8b9/potret-penyandang-disabilitas-di-indonesia--hasil-long-form-sp2020.html
[2] “World report on hearing.” Accessed: Oct. 15, 2025. [Online]. Available: https://www.who.int/publications/i/item/9789240020481
[3] F. Tsina, A. Kusmawati, J. K. Ahmad Dahlan, K. Ciputat Timur, and K. Tangerang Selatan, “Dukungan Sosial Terhadap Kualitas Hidup Kelompok Tuli Di Majelis Ta’lim Tuli Indonesia,” Huk. Inov. J. Ilmu Huk. Sos. dan Hum., vol. 1, no. 2, pp. 71–80, Mar. 2024, doi: 10.62383/HUMIF.V1I2.94.
[4] D. A. Saraswati, V. D. Towidjojo, and Hasanuddin, “Bahasa Isyarat Indonesia,” J. Med. Prof., vol. 4, no. 1, pp. 8–14, 2022, Accessed: Oct. 15, 2025. [Online]. Available: https://jurnal.fk.untad.ac.id/index.php/medpro/article/view/582
[5] E. L. Kelana, M. R. A. Prasetya, . M., and M. Zulfadhilah, “Integrating the CNN Model with the Web for Indonesian Sign Language (BISINDO) Recognition,” J. Appl. Informatics Comput., vol. 9, no. 3, pp. 883–896, Jun. 2025, doi: 10.30871/JAIC.V9I3.9345.
[6] M. Z. Uddin, C. Boletsis, and P. Rudshavn, “Real-Time Norwegian Sign Language Recognition Using MediaPipe and LSTM,” Multimodal Technol. Interact. 2025, Vol. 9, Page 23, vol. 9, no. 3, p. 23, Mar. 2025, doi: 10.3390/MTI9030023.
[7] B. Alsharif, E. Alalwany, and M. Ilyas, “Transfer learning with YOLOV8 for real-time recognition system of American Sign Language Alphabet,” Franklin Open, vol. 8, p. 100165, Sep. 2024, doi: 10.1016/J.FRAOPE.2024.100165.
[8] G. Hugar, R. M. Kagalkar, and A. Das, “Comparative Study of Hybrid Deep Learning Models for Kannada Sign Language Recognition,” Int. J. Comput. Intell. Syst., vol. 18, no. 1, pp. 1–23, Dec. 2025, doi: 10.1007/S44196-025-00922-4/TABLES/7.
[9] B. Alsharif, E. Alalwany, A. Ibrahim, I. Mahgoub, and M. Ilyas, “Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking,” Sensors 2025, Vol. 25, Page 2138, vol. 25, no. 7, p. 2138, Mar. 2025, doi: 10.3390/S25072138.
[10] S. Suherman, A. Suhendra, and E. Ernastuti, “Method Development Through Landmark Point Extraction for Gesture Classification With Computer Vision and MediaPipe,” TEM J., vol. 12, no. 3, pp. 1677–1686, Aug. 2023, doi: 10.18421/TEM123-49.
[11] S. Kamble, “SLRNet: A Real-Time LSTM-Based Sign Language Recognition System,” Jun. 2025, Accessed: Oct. 15, 2025. [Online]. Available: https://arxiv.org/pdf/2506.11154
[12] D. Amalfitano, V. D. ’ Angelo, A. M. Rinaldi, C. Russo, and C. Tommasino, “Enhancing Gesture Recognition for Sign Language Interpretation in Challenging Environment Conditions: A Deep Learning Approach.,” pdfs.semanticscholar.orgD Amalfitano, V D’Angelo, AM Rinaldi, C Russo, C TommasinoKDIR, 2023•pdfs.semanticscholar.org, 2023, doi: 10.5220/0012209700003598.
[13] J. Qin, M. W.-S. Reports, and undefined 2025, “Sign language recognition based on dual-channel star-attention convolutional neural network,” nature.comJ Qin, M WangScientific Reports, 2025•nature.com, Accessed: Oct. 15, 2025. [Online]. Available: https://www.nature.com/articles/s41598-025-13625-9
[14] S. Mehta and M. Rastegari, “Mobilevit: Light-Weight, General-Purpose, and Mobile-Friendly Vision Transformer,” ICLR 2022 - 10th Int. Conf. Learn. Represent., vol. 3, 2022.
[15] K. Meng and K. Chen, “Navigating Efficiency in MobileViT through Gaussian Process on Global Architecture Factors,” Jun. 2024, Accessed: Oct. 15, 2025. [Online]. Available: https://arxiv.org/pdf/2406.04820
[16] F. De et al., “Reducing Computational Cost in MobileViT for Edge-Oriented Models Through Token Merging,” Electron. 2024, Vol. 13, Page 5009, vol. 13, no. 24, p. 5009, Dec. 2024, doi: 10.3390/ELECTRONICS13245009.
[17] M. Zhang et al., “Dual-Attention-Enhanced MobileViT Network: A Lightweight Model for Rice Disease Identification in Field-Captured Images,” Agric., vol. 15, no. 6, pp. 1–22, 2025, doi: 10.3390/agriculture15060571.
[18] “Bisindo - Video Dataset.” Accessed: Oct. 15, 2025. [Online]. Available: https://www.kaggle.com/datasets/rizkyyangpalsu/bisindo-video-dataset
[19] “Learning OpenCV: Computer Vision with the OpenCV Library - Gary Bradski, Adrian Kaehler - Google Books.” Accessed: Oct. 15, 2025.
[20] F. Zhang et al., “MediaPipe Hands: On-device Real-time Hand Tracking,” 2020, [Online]. Available: http://arxiv.org/abs/2006.10214
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Chanan Artamma, Majid Rahardi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
