Neural Machine Translation of Balinese-Indonesian Using T5 Architecture with QLoRA Optimization
DOI:
https://doi.org/10.30871/jaic.v10i3.12771Keywords:
Neural Machine Translation, T5, QloRA, Balinese Language, Low-Resource LanguageAbstract
This study proposes a Neural Machine Translation (NMT) system for Balinese–Indonesian translation by integrating the T5 architecture with Quantized Low-Rank Adaptation (QLoRA) to address low-resource constraints. The model is trained using the NusaTranslation dataset, consisting of 140,972 parallel sentence pairs, and optimized through parameter-efficient fine-tuning with 4-bit quantization and low-rank adaptation. Unlike conventional full fine-tuning, the proposed approach updates only a small fraction of parameters, significantly improving computational efficiency. Experimental results show that the proposed model achieves a BLEU score of 27.93%, ROUGE-1 of 18.94%, ROUGE-2 of 11.96%, ROUGE-L of 18.54%, and BERTScore F1 of 70.49%, indicating competitive performance in lexical, structural, and semantic evaluation aspects. These results demonstrate that QLoRA can maintain translation quality while reducing computational costs. Furthermore, qualitative analysis reveals that the model is capable of generating fluent and contextually appropriate translations, although challenges remain in handling complex sentence structures and linguistic variations. This study highlights the effectiveness of parameter-efficient fine-tuning for low-resource language translation and provides practical implications for developing scalable translation systems for regional languages.
Downloads
References
[1] A. Vaswani et al., “Attention Is All You Need,” 2023.
[2] L. Xue et al., “mT5: A Massively Multilingual Pre-Trained Text-To-Text Transformer,” Mar. 2021, [Online]. Available: http://arxiv.org/abs/2010.11934
[3] T. B. Brown et al., “Language Models are Few-Shot Learners,” Jul. 2020, [Online]. Available: http://arxiv.org/abs/2005.14165
[4] Y. Handayani, Ragam Bahasa di Indonesia. 2019.
[5] J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” May 2019, [Online]. Available: https://github.com/tensorflow/tensor2tensor
[6] C. Raffel et al., “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” Journal of Machine Learning Research, vol. 21, pp. 1–67, 2020, [Online]. Available: http://jmlr.org/papers/v21/20-074.html.
[7] Z. M. Zayyanu, “Revolutionising Translation Technology: A Comparative Study of Variant Transformer Models - BERT, GPT, and T5,” Computer Science & Engineering: An International Journal, vol. 14, no. 3, pp. 15–27, Jun. 2024, doi: 10.5121/cseij.2024.14302.
[8] A. Hannan, S. Kr. Sarma, and Z. Hussain, “Marie A Statistical Approach to Build a Machine Translation System for English Assamese Language Pair,” International Journal of Computer Sciences and Engineering, vol. 7, no. 3, pp. 774–779, Mar. 2019, doi: 10.26438/ijcse/v7i3.774779.
[9] H. Sun and B. Kong, “Sustainable Improvement And Application Of Multilingual English Translation Quality Using T5 and MAML,” Discover Artificial Intelligence, vol. 4, no. 1, Dec. 2024, doi: 10.1007/s44163-024-00213-5.
[10] E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” Oct. 2021, [Online]. Available: http://arxiv.org/abs/2106.09685
[11] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” May 2023, [Online]. Available: http://arxiv.org/abs/2305.14314
[12] D. Heikkinen, D. Sethi, and J. Yiu, Building Transformer Models with Attention. 2022.
[13] D. I. Af’idah and A. Susanto, “Hyperparameter Tuning Seq2Seq Gated Recurrent Unit Untuk Penerjemahan Bahasa Daerah Ke Nasional,” Jurnal Informatika Teknologi dan Sains, vol. 6, pp. 1238–1248, Apr. 2024, doi: 10.51401/jinteks.v6i4.5645.
[14] T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “BERTScore: Evaluating Text Generation with BERT,” Feb. 2020, [Online]. Available: http://arxiv.org/abs/1904.09675
[15] M. S. Maksum, T. Arifin, R. Rohidin, M. A. B. Prasetya, and I. F. Anshori, “Optimalisasi Algoritma Terjemahan Bahasa Dengan Model Transformer: Pendekatan Statistical Machine Learning,” INFOTECH journal, vol. 10, no. 2, pp. 282–287, Aug. 2024, doi: 10.31949/infotech.v10i2.11132.
[16] F. Razsiah, A. Josi, and S. Mubaroh, “Aplikasi Penerjemah Bahasa Bangka Ke Bahasa Indonesia Menggunakan Neural Machine Translation Berbasis Website,” Jurnal Inovasi Teknologi Terapan (JITT), vol. 01, no. 1, Jan. 2023, doi: 10.33504/jitt.v1i1.67.
[17] S. Miyagawa, “Machine Translation for Highly Low-Resource Language: A Case Study of Ainu, a Critically Endangered Indigenous Language in Northern Japan,” Association for Computational Linguistics, pp. 120–124, Dec. 2023, [Online]. Available: https://huggingface.co/SoMiyagawa/
[18] L. J. Laki and Z. G. Yang, “Neural Machine Translation for Hungarian,” Acta Linguistica Academica, vol. 69, no. 4, pp. 501–520, Dec. 2022, doi: 10.1556/2062.2022.00576.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Leonard Kumaro, I Gusti Ngurah Lanang Wijayakusuma, IPW Gautama
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
