Sentiment Classification of Indonesian E-Government Application Reviews Using Advanced Learning Models
DOI:
https://doi.org/10.30871/jaic.v10i2.12217Keywords:
E-Government Reviews, Learning Models, Natural Language, Sentiment Analysis, Text ClassificationAbstract
The digital transformation of public services in Indonesia has led to the development of e-government applications such as Cek Bansos, aimed at improving transparency in social assistance distribution. However, user reviews indicate varying perceptions of service quality. This study conducts a comparative evaluation of machine learning and deep learning models for sentiment classification of Indonesian e-government application reviews. A total of 28,697 reviews were collected via web scraping, with 27,985 retained after preprocessing. Sentiment labels were assigned automatically based on rating scores (1–2 as negative, 4–5 as positive), while neutral reviews were excluded. To address class imbalance, SMOTE and Random Oversampling were applied to the training data for machine learning and deep learning models, respectively. TF-IDF features were used with Logistic Regression, Support Vector Machine, and Random Forest, while word embeddings were implemented with CNN, BiLSTM, and BiGRU. Results show that BiLSTM achieved the highest accuracy (85.71%), whereas Logistic Regression obtained the highest F1-score (0.7975). The small performance gap (<2%) indicates that traditional machine learning models remain competitive with deep learning approaches under statistically comparable performance. This study provides empirical evidence in the Indonesian e-government context and offers practical insights for monitoring public feedback to improve digital public services.
Downloads
References
[1] N. Desripa et al., “Volume 2 ; Nomor 2 ; Februari 2024,” pp. 7–12, doi: 10.59435/gjmi.v2i2.275.
[2] M. H. Asqalani and B. Fitanto, “Analisis Determinan Tingkat Kemiskinan Pada Masa Pre-Post Pandemi Covid-19 Di Provinsi Jawa Timur,” Journal of Development Economic and Social Studies, vol. 3, no. 4, pp. 1098–1114, Oct. 2024, doi: 10.21776/jdess.2024.03.4.10.
[3] S. J. A. Samuda and E. Suprihartiningsih, “COVID-19 Social Assistance Program and Poverty: Evidence from Indonesia,” Jurnal Ekonomi Pembangunan, vol. 20, no. 2, pp. 125–134, Jan. 2023, doi: 10.29259/jep.v20i2.19088.
[4] A. Rahma et al., “Menggali Akar Permasalahan: Kajian Mendalam Terhadap Data Kemiskinan Dan Mekanisme Bantuan Sosial,” Jurnal Penelitian Nusantara, vol. 1, pp. 192–198, 2025, doi: 10.59435/menulis.v1i3.93.
[5] Dwiarto, Raden. "Inovasi Penyaluran Jaminan Sosial Tepat Sasaran Melalui Kebijakan Pengelolaan Anggaran Data Terpadu Kesejahteraan Sosial (DTKS) Dan Pemanfaatan Aplikasi" Cek Bansos"." Prosiding Seminar Nasional Unimus. Vol. 6. 2023.
[6] M. Khanif Hidayatul Khasan, T. Riasih, and E. Gunawan Wibisono, “Persepsi Warga Masyarakat Terhadap Aplikasi Cek Bansos Dalam Penyaluran Bantuan Sosial Di Desa Kalikidang Kecamatan Sokaraja Kabupaten Banyumas Jawa Tengah,” 2024, doi: 10.31595/10.31595/lindayasos.v7i1.1604.
[7] D. Salsabila, D. Chusnulitta Jatnika, and F. P. Firsanty, “Focus : Jurnal Pekerjaan Sosial Pemanfaatan Teknologi Digital Dalam Layanan Sosial Di Indonesia: Tinjauan Sistematis,” vol. 8, no. 1, pp. 50–59, 2025, doi: 10.24198/focus.v8i1.63672.
[8] T. Shaik, X. Tao, C. Dann, H. Xie, Y. Li, and L. Galligan, “Sentiment analysis and opinion mining on educational data: A survey,” Natural Language Processing Journal, vol. 2, p. 100003, Mar. 2023, doi: 10.1016/j.nlp.2022.100003.
[9] Q. A. Xu, V. Chang, and C. Jayne, “A systematic review of social media-based sentiment analysis: Emerging trends and challenges,” Decision Analytics Journal, vol. 3, p. 100073, Jun. 2022, doi: 10.1016/j.dajour.2022.100073.
[10] A. Noor, M. D. Mehmood, and T. Das, “End Users’ Perspective of Performance Issues in Google Play Store Reviews,” in Product-Focused Software Process Improvement, D. Taibi, M. Kuhrmann, T. Mikkonen, J. Klünder, and P. Abrahamsson, Eds., Cham: Springer International Publishing, 2022, pp. 603–609.
[11] A. Yasin, R. Fatima, A. N. Ghazi, and Z. Wei, “Python data odyssey: Mining user feedback from google play store,” Data Brief, vol. 54, Jun. 2024, doi: 10.1016/j.dib.2024.110499.
[12] J. R. Jim, M. A. R. Talukder, P. Malakar, M. M. Kabir, K. Nur, and M. F. Mridha, “Recent advancements and challenges of NLP-based sentiment analysis: A state-of-the-art review,” Mar. 01, 2024, Elsevier Ltd. doi: 10.1016/j.nlp.2024.100059.
[13] N. Malik and M. Bilal, “Natural language processing for analyzing online customer reviews: a survey, taxonomy, and open research challenges,” PeerJ Comput Sci, vol. 10, 2024, doi: 10.7717/PEERJ-CS.2203.
[14] Israt Jahan, Md Nakibul Islam, Md Mahadi Hasan, and Md Rafiuddin Siddiky, “Comparative analysis of machine learning algorithms for sentiment classification in social media text,” World Journal of Advanced Research and Reviews, vol. 23, no. 3, pp. 2842–2852, Sep. 2024, doi: 10.30574/wjarr.2024.23.3.2983.
[15] S. R. Putri et al., “Analisis Sentimen Publik terhadap Nadiem Makarim sebagai Mendikbudrisktek menggunakan Support Vector Machine (SVM).” Available: http://sistemasi.ftik.unisi.ac.id
[16] Z. Gao, Z. Li, J. Luo, and X. Li, “Short Text Aspect-Based Sentiment Analysis Based on CNN + BiGRU,” Applied Sciences (Switzerland), vol. 12, no. 5, Mar. 2022, doi: 10.3390/app12052707.
[17] A. S. Talaat, “Sentiment analysis classification system using hybrid BERT models,” J Big Data, vol. 10, no. 1, Dec. 2023, doi: 10.1186/s40537-023-00781-w.
[18] L. Ashbaugh and Y. Zhang, “A Comparative Study of Sentiment Analysis on Customer Reviews Using Machine Learning and Deep Learning,” Computers, vol. 13, no. 12, Dec. 2024, doi: 10.3390/computers13120340.
[19] C. P. Chai, “Comparison of text preprocessing methods,” Nat Lang Eng, vol. 29, no. 3, pp. 509–553, May 2023, doi: 10.1017/S1351324922000213.
[20] M. A. Palomino and F. Aider, “Evaluating the Effectiveness of Text Pre-Processing in Sentiment Analysis,” Applied Sciences (Switzerland), vol. 12, no. 17, Sep. 2022, doi: 10.3390/app12178765.
[21] M. Kumar, L. Khan, and H. T. Chang, “Evolving techniques in sentiment analysis: a comprehensive review,” 2025, PeerJ Inc. doi: 10.7717/PEERJ-CS.2592.
[22] V. Gupta and P. Rattan, “Advancing Sentiment Analysis in Restaurant Reviews through Unsupervised Machine Learning Algorithms,” International Journal of Intelligent Engineering and Systems, vol. 17, no. 4, pp. 1108–1121, 2024, doi: 10.22266/IJIES2024.0831.82.
[23] N. Jiang, C. Luo, V. Lakshman, Y. Dattatreya, and Y. Xue, “Massive Text Normalization via an Efficient Randomized Algorithm,” in WWW 2022 - Proceedings of the ACM Web Conference 2022, Association for Computing Machinery, Inc, Apr. 2022, pp. 2946–2956. doi: 10.1145/3485447.3512015.
[24] K. S. Eljil, F. Nait-Abdesselam, E. Hamouda, and M. Hamdi, “Enhancing Sentiment Analysis on Social Media with Novel Preprocessing Techniques,” Journal of Advances in Information Technology, vol. 14, no. 6, pp. 1206–1213, 2023, doi: 10.12720/jait.14.6.1206-1213.
[25] H. Dwiharyono and S. Suyanto, “Stemming for Better Indonesian Text-to-Phoneme,” Ampersand, vol. 9, Jan. 2022, doi: 10.1016/j.amper.2022.100083.
[26] L. Zhang, “Features extraction based on Naive Bayes algorithm and TF-IDF for news classification,” PLoS One, vol. 20, no. 7 July, Jul. 2025, doi: 10.1371/journal.pone.0327347.
[27] K. Yusupov, M. R. Islam, I. Muminov, M. Sahlabadi, and K. Yim, “Comparative Analysis of Machine Learning and Deep Learning Models for Email Spam Classification Using TF-IDF and Word Embedding Techniques,” in Advances on Broad-Band Wireless Computing, Communication and Applications, L. Barolli, Ed., Cham: Springer Nature Switzerland, 2025, pp. 114–122.
[28] P. Sankar, N. Palanichamy, and K. W. Ng, “Sentiment Analysis on Twitter Data for Depression Detection,” Journal of Logistics, Informatics and Service Science, vol. 11, no. 3, pp. 21–36, 2024, doi: 10.33168/JLISS.2024.0302.
[29] A. Rajesh and T. Hiwarkar, “Sentiment analysis from textual data using multiple channels deep learning models,” Journal of Electrical Systems and Information Technology, vol. 10, no. 1, Nov. 2023, doi: 10.1186/s43067-023-00125-x.
[30] M. Hayaeian Shirvan, M. H. Moattar, and M. Hosseinzadeh, “Deep generative approaches for oversampling in imbalanced data classification problems: A comprehensive review and comparative analysis,” Feb. 01, 2025, Elsevier Ltd. doi: 10.1016/j.asoc.2024.112677.
[31] C. Suhaeni and H. S. Yong, “Mitigating Class Imbalance in Sentiment Analysis through GPT-3-Generated Synthetic Sentences,” Applied Sciences (Switzerland), vol. 13, no. 17, Sep. 2023, doi: 10.3390/app13179766.
[32] N. Alturayeif and J. Hassine, “Data leakage detection in machine learning code: transfer learning, active learning, or low-shot prompting?,” PeerJ Comput Sci, vol. 11, 2025, doi: 10.7717/peerj-cs.2730.
[33] P. Pramanik, S. Samanta, R. K. Mondal, J. Patra, U. Adhikari, and S. Gupta, “Enhancing Text Intelligence with Soft Voting and TF-IDF Logistic Learners,” in Proceedings of Data Analytics and Management, A. Swaroop, B. Virdee, S. D. Correia, and Z. Polkowski, Eds., Cham: Springer Nature Switzerland, 2026, pp. 301–313.
[34] A. Alqurafi and T. Alsanoosy, “Measuring Customers’ Satisfaction Using Sentiment Analysis: Model and Tool,” Journal of Computer Science, vol. 20, no. 4, pp. 419–430, 2024, doi: 10.3844/jcssp.2024.419.430.
[35] S. U. Hassan, J. Ahamed, and K. Ahmad, “Analytics of machine learning-based algorithms for text classification,” Sustainable Operations and Computers, vol. 3, pp. 238–248, Jan. 2022, doi: 10.1016/j.susoc.2022.03.001.
[36] S. E. Sorour, A. Alojail, A. El-Shora, A. E. Amin, and A. A. Abohany, “A Hybrid Deep Learning Approach for Enhanced Sentiment Classification and Consistency Analysis in Customer Reviews,” Mathematics, vol. 12, no. 23, Dec. 2024, doi: 10.3390/math12233856.
[37] Y. Mao, Y. Zhang, L. Jiao, and H. Zhang, “Document-Level Sentiment Analysis Using Attention-Based Bi-Directional Long Short-Term Memory Network and Two-Dimensional Convolutional Neural Network,” Electronics (Switzerland), vol. 11, no. 12, Jun. 2022, doi: 10.3390/electronics11121906.
[38] D. Pandya and A. Thakkar, “Sentiment Analysis of Self Driving Car Dataset: A comparative study of Deep Learning approaches,” in Procedia Computer Science, Elsevier B.V., 2024, pp. 12–21. doi: 10.1016/j.procs.2024.04.002.
[39] E. Altuncu, V. N. L. Franqueira, and S. Li, “Deepfake: definitions, performance metrics and standards, datasets, and a meta-review,” 2024, Frontiers Media SA. doi: 10.3389/fdata.2024.1400024.
[40] M. C. Hinojosa Lee, J. Braet, and J. Springael, “Performance Metrics for Multilabel Emotion Classification: Comparing Micro, Macro, and Weighted F1-Scores,” Applied Sciences (Switzerland), vol. 14, no. 21, Nov. 2024, doi: 10.3390/app14219863.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Aulia Diaz Gustiavani, Muljono Muljono
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
