Comparative Sentiment Analysis and Topic Modeling of Dola AI User Reviews for Knowledge Discovery
DOI:
https://doi.org/10.30871/jaic.v10i5.13834Keywords:
Sentiment Analysis, Knowledge Discovery, Machine Learning, Latent Dirichlet Allocation, Dola AIAbstract
As artificial intelligence (AI) technologies become more widely adopted, users increasingly express their experiences through online reviews, providing valuable insights into user satisfaction and opportunities for application improvement. This study compares the performance of Support Vector Machine (SVM), Naïve Bayes (NB), and K-Nearest Neighbor (KNN) for sentiment classification of Dola AI user reviews and applies Latent Dirichlet Allocation (LDA) to identify the dominant topics within each sentiment category. The research followed the Cross-Industry Standard Process for Data Mining (CRISP-DM) framework using 28,226 Google Play Store reviews collected through web scraping. The reviews were represented using Term Frequency–Inverse Document Frequency (TF-IDF), class imbalance was addressed using the Synthetic Minority Oversampling Technique (SMOTE), and model performance was evaluated through 5-fold stratified cross-validation using accuracy, weighted precision, weighted recall, and weighted F1-score. The results showed that SVM achieved the highest average performance, with an accuracy of 77.68% and a weighted F1-score of 80.02%, followed by Naïve Bayes with a weighted F1-score of 78.60% and K-Nearest Neighbor with 70.42%. Furthermore, LDA identified six topics in positive reviews, three topics in neutral reviews, and two topics in negative reviews. Positive reviews mainly highlighted the application's ease of use, AI capabilities, and conversational features, whereas negative reviews primarily focused on application performance, the quality of AI-based image editing, and advertisements. These findings provide deeper insights into user perceptions and offer data-driven recommendations for improving the quality and user experience of the Dola AI application.
Downloads
References
[1] F. F.-H. Nah, R. Zheng, J. Cai, K. Siau, and L. Chen, “Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration,” J. Inf. Technol. Case Appl. Res., vol. 25, no. 3, pp. 277–304, 2023, doi: 10.1080/15228053.2023.2233814.
[2] R. Alabduljabbar, “User-centric AI: evaluating the usability of generative AI applications through user reviews on app stores,” PeerJ Comput. Sci., vol. 10, 2024, doi: 10.7717/peerj-cs.2421.
[3] D. H. Hagos, R. Battle, and D. B. Rawat, “Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives,” IEEE Trans. Artif. Intell., vol. 5, no. 12, pp. 5873–5893, Dec. 2024, doi: 10.1109/TAI.2024.3444742.
[4] A. Iorliam and J. A. Ingio, “A Comparative Analysis of Generative Artificial Intelligence Tools for Natural Language Processing,” J. Comput. Theor. Appl., vol. 1, no. 3, pp. 311–325, 2024, doi: 10.62411/jcta.9447.
[5] O. N. Salsabila, Navigasi Dola AI - Smart Assistant: Menjelajahi Arsitektur Kecerdasan Buatan Dalam Ruang Komunikasi Digital, 1st ed. Karawang: Afdan Rojabi Publisher, 2026.
[6] M. Alghareeb, A. S. Albesher, and A. Asif, “Studying Users’ Perceptions of COVID-19 Mobile Applications in Saudi Arabia,” Sustainability, vol. 15, no. 2, pp. 1–17, 2023, doi: 10.3390/su15020956.
[7] T. N. Prakash and A. Aloysius, “Textual Sentiment Analysis using Lexicon Based Approaches,” Ann. Rom. Soc. Cell Biol., vol. 25, no. 4, pp. 9878–9885, 2021.
[8] A. R. Makhtum and M. Muhajir, “Sentiment Analysis of Omnibus Law Using Support Vector Machine (Svm) With Linear Kernel,” Barekeng, vol. 17, no. 4, pp. 2197–2206, 2023, doi: 10.30598/barekengvol17iss4pp2197-2206.
[9] T. Kim and J. S. Lee, “Exponential Loss Minimization for Learning Weighted Naive Bayes Classifiers,” IEEE Access, vol. 10, pp. 22724–22736, 2022, doi: 10.1109/ACCESS.2022.3155231.
[10] R. K. Halder, M. N. Uddin, M. A. Uddin, S. Aryal, and A. Khraisat, “Enhancing K-nearest neighbor algorithm: a comprehensive review and performance analysis of modifications,” J. Big Data, vol. 11, no. 1, pp. 1–55, 2024, doi: 10.1186/s40537-024-00973-y.
[11] M. M. A. Pratama, D. Kurniawan, A. Rifai, and K. D. Tania, “Analisis Sentimen dan Topik Ulasan Aplikasi Komunitas Digital Gamer dengan SVM - LDA dan CRISP - DM,” vol. 15, no. 1, pp. 181–193, 2026, doi: 10.32520/stmsi.v15i1.5746.
[12] N. A. S. ER, “Implementasi Latent Dirichlet Allocation (Lda) Untuk Klasterisasi Cerita Berbahasa Bali,” J. Teknol. Inf. Ilmu Komput., vol. 8, no. 1, pp. 127–134, 2021, doi: 10.25126/jtiik.202183556.
[13] S. Nurmalasari, A. N. Hidayanto, L. A. Huwaida, and H. Wulandari, “Sentiment Analysis and Topic Modeling of Citizen Satisfaction with the Indonesian Government in Handling a Pandemic,” OPSearch Am. J. Open Res., vol. 2, no. 7, pp. 246–256, 2023, doi: 10.58811/opsearch.v2i6.61.
[14] S. A. Putri, K. D. Tania, and Naretha Kawadha Pasemah Gumay, “Knowledge Discovery Through Sentiment Analysis and Topic Modeling of BCA Mobile and MyBCA,” J. Math. Comput. Stat., vol. 8, no. 2, pp. 669–682, 2025, doi: 10.35580/jmathcos.v8i2.9782.
[15] D. K. Harahap, K. D. Tania, and P. E. Sevtiyuni, “Topic Mining-Based Knowledge Discovery of User Health Information Needs,” Indones. J. Electron. Electromed. Eng. Med. Informatics, vol. 7, no. 4, pp. 641–653, 2025, doi: https://doi.org/10.35882/ijeeemi.v7i4.270.
[16] M. R. Maulizidan and K. D. Tania, “Knowledge Discovery Based on Sentiment Analysis of Public Perceptions About Generative AI on X,” Indones. J. Informatics Educ., vol. 9, no. 2, pp. 92–107, 2025, doi: http://dx.doi.org/10.20961/ijie.v9i2.107758.
[17] W. Ningsih, B. Alfianda, R. Rahmaddeni, and D. Wulandari, “Perbandingan Algoritma SVM dan Naïve Bayes dalam Analisis Sentimen Twitter pada Penggunaan Mobil Listrik di Indonesia,” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 4, no. 2, pp. 556–562, 2024, doi: 10.57152/malcom.v4i2.1253.
[18] R. K. Das, M. Islam, M. M. Hasan, S. Razia, M. Hassan, and S. A. Khushbu, “Sentiment analysis in multilingual context: Comparative analysis of machine learning and hybrid deep learning models,” Heliyon, vol. 9, no. 9, pp. 1–22, 2023, doi: 10.1016/j.heliyon.2023.e20281.
[19] M. E. M. Abo et al., “A multi-criteria approach for arabic dialect sentiment analysis for online reviews: Exploiting optimal machine learning algorithm selection,” Sustainability, vol. 13, no. 18, pp. 1–20, 2021, doi: 10.3390/su131810018.
[20] M. Hanafi and M. Furqan, “Perbandingan Analisis Sentimen Presiden 2024 Menggunakan Algoritma Support Vector Machine dan K-Nearest Neighbor,” CESS (Journal Comput. Eng. Syst. Sci., vol. 10, no. 1, pp. 275–285, 2025, doi: 10.24114/cess.v10i1.67747.
[21] C. Schröer, F. Kruse, and J. M. Gómez, “A systematic literature review on applying CRISP-DM process model,” Procedia Comput. Sci., vol. 181, pp. 526–534, 2021, doi: 10.1016/j.procs.2021.01.199.
[22] M. R. Maulana, K. D. Tania, and P. Sari, “Comparative Analysis of Machine Learning Models for Daily Urban Waste Volume’s Knowledge Discovery,” Int. Conf. Artif. Intell. Technol. Solut., pp. 131–136, 2025, doi: 10.1109/ICAITech66481.2025.11387653.
[23] Rudianto, T. I. Solihati, and R. Kania, Data Mining Untuk Analisis Prilaku dan Kecerdasan Bisnis, 1st ed. Banten: PT Literapedia Utama, 2025.
[24] A. Sánchez, C. Vidal-Silva, G. Mancilla, M. Tupac-Yupanqui, and J. M. Rubio, “Sustainable e-Learning by Data Mining—Successful Results in a Chilean University,” Sustainability, vol. 15, no. 2, pp. 1–16, 2023, doi: 10.3390/su15020895.
[25] A. Nursikuwagus, Suherman, H. Purwanto, and T. Hartono, “SUPPORT VECTOR MACHINE TO CLASSIFY SENTIMENT REVIEWS ON GOOGLE PLAY STORE,” JITK, vol. 11, no. 3, pp. 724–732, 2022, doi: 10.33480/jitk.v11i3.7282.kemampuan.
[26] S. Szeghalmy and A. Fazekas, “A Comparative Study of the Use of Stratified Cross-Validation and Distribution-Balanced Stratified Cross-Validation in Imbalanced Learning,” Sensors, vol. 23, no. 4, pp. 1–27, 2023, doi: 10.3390/s23042333.
[27] A. Setiawan, R. Yanti, E. Ali, and H. Yenni, “Optimasi Klasifikasi Tingkat Obesitas Pada Remaja Berdasarkan Pola Hidup Menggunakan SVM Dengan Teknik Smote,” J. Algoritm., vol. 22, no. 2, pp. 224–235, 2025, doi: 10.33364/algoritma/v.22-2.2509.
[28] R. Andriyanto and K. Kusrini, “Enhancing Sentiment Analysis Effectiveness with LSTM Variants, and Stratified K-Fold on Imbalanced Dataset,” Build. Informatics, Technol. Sci., vol. 6, no. 2, pp. 981–990, 2024, doi: 10.47065/bits.v6i2.5721.
[29] M. Rudiger, D. Antons, A. M. Joshi, and T.-O. Salge, “Topic modeling revisited : New evidence on algorithm performance and quality metrics,” PLoS One, vol. 17, no. 4, pp. 1–25, 2022, doi: 10.1371/journal.pone.0266325.
[30] U. Fayyad and R. Uthurusamy, “Data Mining and Knowledge Discovery in Databases,” Commun. ACM, vol. 39, no. 11, pp. 24–26, 1996, doi: 10.1145/240455.240463.
[31] M. Nashir, Dian Ade Kurnia, Yudhistira Arie Wijaya, Ade Irma Purnama Sari, and Nisa Dienwati Nuris, “Perbandingan Kinerja Svm Dan Naïve Bayes Pada Analisis Sentimen Komentar Demonstrasi Dpr 25 Agustus 2025,” J. Mhs. Sist. Inf., vol. 7, no. 1, pp. 392–401, 2025, doi: 10.24127/jmsi.v7i1.10650.
[32] Y. A. Wahid, Sanatang, and D. D. Andayani, “Performance Comparison of Svm and Naïve Bayes For Indonesian-Language Sentiment Analysis On Free Fire Reviews Using Tf-Idf And Smote,” J. Embed. Syst. Secur. Intell. Syst., vol. 6, no. 4, pp. 674–689, 2025, doi: 10.59562/jessi.v6i4.10818.
[33] A. R. Noor and D. Normawati, “Perbandingan Support Vector Machine Dan K-Nearest Neighbor Pada Analisis Sentimen Pelaksanaan Pemilihan Presiden Tahun 2024,” JIPI (Jurnal Ilm. Penelit. dan Pembelajaran Inform., vol. 11, no. 2, pp. 1021–1030, 2026, doi: 10.29100/jipi.v11i2.8120.
[34] Y. Zhai, X. Song, Y. Chen, and W. Lu, “A Study of Mobile Medical App User Satisfaction Incorporating Theme Analysis and Review Sentiment Tendencies,” Int. J. Environ. Res. Public Health, vol. 19, no. 12, 2022, doi: 10.3390/ijerph19127466.
[35] Endar Nirmala and Andri Fahmi, “Comparison of LSTM and Naïve Bayes in Google Play Store App Review Sentiment Analysis,” J. Inotera, vol. 11, no. 1, pp. 171–181, 2026, doi: 10.31572/inotera.vol11.iss1.2026.id653.
[36] E. Ernest Mfumbilwa, C. Mnong’one, E. Chao, and D. Amani, “Invigorating continuance intention among users of AI chatbots in the banking industry: an empirical study from Tanzania,” Cogent Bus. Manag., vol. 11, no. 1, pp. 1–17, 2024, doi: 10.1080/23311975.2024.2419482.
[37] C. Gao, J. Zeng, D. Lo, X. Xia, I. King, and M. R. Lyu, “Understanding in-app advertising issues based on large scale app review analysis,” Inf. Softw. Technol., vol. 142, pp. 1–13, 2022, doi: 10.1016/j.infsof.2021.106741.
[38] P. Shrivastava, “Understanding acceptance and resistance toward generative AI technologies: a multi-theoretical framework integrating functional, risk, and sociolegal factors,” Front. Artif. Intell., vol. 8, no. April, pp. 1–10, 2025, doi: 10.3389/frai.2025.1565927.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Nadia Ramadhani, Ken Ditha Tania, Naretha Kawadha Pasemah Gumay
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
