Adaptive Retrieval-Augmented Generation with Domain Specific Fine Tuning For Smart MSME Digital Transformation
DOI:
https://doi.org/10.30871/jaic.v10i3.12924Keywords:
Adaptive Retrieval-Augmented Generation (ARAG), Chatbot, digital transformation, IndoBERT, MSMEs, WhatsAppAbstract
Micro, Small, and Medium Enterprises (MSMEs) play a significant role in the Indonesian economy; however, the adoption of digital technologies among MSMEs remains limited, reducing operational efficiency and business competitiveness. This study proposes an Adaptive Retrieval-Augmented Generation (ARAG) framework integrated with WhatsApp to support MSME digital transformation through contextual conversational AI assistance. The proposed system combines adaptive retrieval mechanisms with domain-specific fine-tuning using IndoBERT and a knowledge base containing 50,000 MSME operational documents. A mixed-methods approach was employed, consisting of system development, comparative evaluation, and field validation involving 200 MSMEs. Experimental results demonstrated that ARAG achieved an average response accuracy of 86.80%, outperforming rule-based, TF-IDF, and generic large language model baselines. The system also achieved a Retrieval Precision@5 of 0.874, an end-to-end F1-score of 0.841, and a lower hallucination rate compared to generic LLM approaches. Field validation showed a 22.7% improvement in operational efficiency, a 17.4% increase in digital adoption rates, and a System Usability Scale (SUS) score of 84.6, categorized as excellent usability. The findings indicate that retrieval grounding and domain adaptation contribute substantially to improving contextual relevance and practical usability in MSME-oriented conversational AI systems. Therefore, the proposed ARAG framework demonstrates strong potential as a practical and scalable digital assistance solution for supporting Indonesian MSME digital transformation.
Downloads
References
[1] Kementerian Koperasi dan UKM. Laporan Tahunan UMKM 2023. Jakarta; 2023.
[2] Institute for Development of Economics and Finance (INDEF). Peran Platform Digital terhadap Pengembangan UMKM di Indonesia. Jakarta; 2024.
[3] Prihandono D, Wijaya AP, Wiratama B, Prananta W, Widia S. Digital transformation to enhance Indonesian SME performance: Exploring the impact of market competition and digital strategy. Problems and Perspectives in Management. 2024;22(2):103–13.
[4] Fatoni MH. Leveled Managerial Training of Central Java Cooperative and Micro, Small and Medium Enterprises (MSMEs) Training Center: Key to Success of Central Java MSMEs Upgrading. Journal of Social Entrepreneurship and Creative Technology. 2024;1(2).
[5] Rinaldi F, Maarif S, Thamrin S, Andang A. Role of Micro, Small, and Medium Enterprises (MSMEs) in Supporting National Defense from Economic Perspective. Journal of Positive School Psychology. 2022;6(5):8914–20.
[6] Heo S, Na S. Ready for departure: Factors to adopt large language model (LLM)-based artificial intelligence (AI) technology in the architecture, engineering and construction (AEC) industry. Results in Engineering. 2025;25.
[7] Nee CK, Rahman MHA, Yahaya N, Ibrahim NH, Razak RA, Sugino C. Exploring the Trend and Potential Distribution of Chatbot in Education: A Systematic Review. International Journal of Information and Education Technology. 2023;13(3):516–25.
[8] Mehta R, Verghese J, Mahajan S, et al. Consumers' behavior in conversational commerce marketing based on messenger chatbots. F1000Research. 2022;11:647.
[9] Adam M, Wessel M, Benlian A. AI-based chatbots in customer service and their effects on user compliance. Electronic Markets. 2021;31(2):427–45.
[10] Lee MK, Neumann JL. Exploring the role of Large Language Model (LLM)-based Chatbots for Human Resources. Austin; 2023.
[11] Sufi F, Alsulami M. AI-Driven Chatbot for Real-Time News Automation. Mathematics. 2025;13(5).
[12] Pawlik L. How the Choice of LLM and Prompt Engineering Affects Chatbot Effectiveness. Electronics. 2025;14(5).
[13] Lim Y, Lim J, Cho N. An Experimental Comparison of the Usability of Rule-based and Natural Language Processing-based Chatbots. Asia Pacific Journal of Information Systems. 2020;30(4):832–46.
[14] Lewis P, Perez E, Piktus A, et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401. 2020.
[15] Zhou Q, Liu C, Duan Y, et al. GastroBot: a Chinese gastrointestinal disease chatbot based on the retrieval-augmented generation. Frontiers in Medicine. 2024;11.
[16] Arslan M, Munawar S, Cruz C. Business insights using RAG–LLMs: a review and case study. Journal of Decision Systems. 2024.
[17] Dogan O, Gurcan OF. Enhancing E-Business Communication with a Hybrid Rule-Based and Extractive-Based Chatbot. Journal of Theoretical and Applied Electronic Commerce Research. 2024;19(3):1984–99.
[18] Zhang L, Yang Y, Zhou J, Chen C, He L. Retrieval-Polished Response Generation for Chatbot. IEEE Access. 2020;8:123882–90.
[19] Ait-Mlouk A, Jiang L. KBot: A Knowledge Graph Based ChatBot for Natural Language Understanding over Linked Data. IEEE Access. 2020;8:149220–30.
[20] Kojima T, Gu SS, Reid M, Matsuo Y, Iwasawa Y. Large Language Models are Zero-Shot Reasoners. arXiv:2205.11916. 2022.
[21] Huang SM, Soepriyanto G, Wahyuningtias D, Winoto A. Social Media Applications for MSMEs in the Era of the Digital Economy. Journal of Theoretical and Applied Information Technology. 2023;15(7).
[22] Mash R, Schouw D, Fischer AE. Evaluating the Implementation of the GREAT4Diabetes WhatsApp Chatbot to Educate People with Type 2 Diabetes during the COVID-19 Pandemic: Convergent Mixed Methods Study. JMIR Diabetes. 2022;7(2).
[23] Bratić D, Šapina M, Jurečić D, Žiljak Gršić J. Centralized Database Access: Transformer Framework and LLM/Chatbot Integration-Based Hybrid Model. Applied System Innovation. 2024;7(1).
[24] Sagstad MH, Morken NH, Lund A, et al. Quantitative User Data From a Chatbot Developed for Women With Gestational Diabetes Mellitus: Observational Study. JMIR Formative Research. 2022;6(4).
[25] Creswell JW, Creswell JD. Research design: Qualitative, quantitative, and mixed methods approaches. 6th ed. Thousand Oaks: SAGE Publications; 2023.
[26] Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research on teaching. 2nd ed. Boston: Houghton Mifflin; 2022.
[27] Tashakkori A, Teddlie C. SAGE handbook of mixed methods in social & behavioral research. 3rd ed. Thousand Oaks: SAGE Publications; 2021.
[28] Zhang Y, Chen X, Wang L. Knowledge base construction for domain-specific chatbots: A systematic approach. Expert Systems with Applications. 2023;210:118434.
[29] Koto F, Rahimi A, Lau JH, Baldwin T. IndoBERT: A pre-trained language model for Indonesian language understanding and generation. arXiv:2009.09745. 2020.
[30] Adamopoulou E, Moussiades L. An overview of chatbot technology. Artificial Intelligence Applications and Innovations. 2020;584:373-383.
[31] Diederich S, Brendel AB, Kolbe LM. Designing anthropomorphic enterprise conversational agents. Business & Information Systems Engineering. 2020;62(3):193-209.
[32] Radziwill NM, Benton MC. Evaluating quality of chatbots and intelligent conversational agents. arXiv:1704.04579. 2017.
[33] Eldridge SM, Chan CL, Campbell MJ, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239.
[34] Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332.
[35] Venkatesh V, Brown SA, Sullivan YW. Guidelines for conducting mixed-methods research: An extension and illustration. Journal of the Association for Information Systems. 2016;17(7):435-494.
[36] Field A. Discovering statistics using IBM SPSS statistics. 5th ed. London: SAGE Publications; 2018.
[37] Barnes RM. Motion and time study: Design and measurement of work. 7th ed. New York: John Wiley & Sons; 1980.
[38] Angrist JD, Pischke JS. Mostly harmless econometrics: An empiricist's companion. Princeton: Princeton University Press; 2009.
[39] Singer JD, Willett JB. Applied longitudinal data analysis: Modeling change and event occurrence. Oxford: Oxford University Press; 2003.
[40] Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3(2):77-101.
[41] Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.
[42] Brynjolfsson E, Hitt LM. Beyond computation: Information technology, organizational transformation and business performance. Journal of Economic Perspectives. 2000;14(4):23-48.
[43] OECD. SME and Entrepreneurship Policy in Indonesia 2018. OECD Studies on SMEs and Entrepreneurship. Paris: OECD Publishing; 2018.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Mawar Hardiyanti, Sri Hartati Wijono, Dwi Poetra Sedjati
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
