Comparison of the Performance of K-Nearest Neighbor and Naive Bayes Algorithms for Sentiment Analysis of PinjamYuk Application User Reviews Using SMOTE and TF-IDF

Lindya Rossita Handoko; Amelia Faza; Mula Agung Barata; Afril Efan Pajri

doi:10.30871/jaic.v10i4.13398

Authors

Lindya Rossita Handoko Universitas Nahdlatul Ulama Sunan Giri Bojonegoro (UNUGIRI)
Amelia Faza Universitas Nahdlatul Ulama Sunan Giri Bojonegoro (UNUGIRI)
Mula Agung Barata Universitas Nahdlatul Ulama Sunan Giri Bojonegoro (UNUGIRI)
Afril Efan Pajri Universitas Nahdlatul Ulama Sunan Giri Bojonegoro (UNUGIRI)

DOI:

https://doi.org/10.30871/jaic.v10i4.13398

Keywords:

K-Nearest Neighbor, Naive Bayes, Sentiment Analysis, SMOTE

Abstract

The growth of online lending services has driven the increasing adoption of digital financial applications, providing users with convenient access to financial services. This growing number of users has generated a large volume of reviews on the Google Play Store, which can serve as a valuable source for understanding users’ perspectives on the benefits and performance of these applications. This study aims to compare the performance of the K-Nearest Neighbor (KNN) and Naive Bayes algorithms in classifying the sentiment of user reviews of the PinjamYuk application. The study uses a secondary dataset obtained from Kaggle, consisting of 500 user reviews of the PinjamYuk application on the Google Play Store during the 2023–2024 period. The reviews were categorized into three sentiment classes: positive, neutral, and negative, based on their rating scores. Because the class distribution in the dataset was imbalanced, the Synthetic Minority Oversampling Technique (SMOTE) was applied to balance the classes before the classification process. The research procedure included data preprocessing, feature extraction using Term Frequency–Inverse Document Frequency (TF-IDF), data balancing using SMOTE, and KNN parameter optimization using GridSearchCV. The models were evaluated using accuracy, precision, recall, F1-score, confusion matrix, stratified K-fold cross-validation, and the McNemar test. The results show that the Naive Bayes algorithm outperformed KNN. The stratified K-fold cross-validation results yielded an average accuracy of 81.0% for Naive Bayes and 80.8% for KNN. Furthermore, the McNemar test produced a p-value of 0.014 (p < 0.05), indicating that the performance difference between the two algorithms was statistically significant. These findings demonstrate that the Naive Bayes algorithm is more effective for analyzing user sentiment in reviews of the PinjamYuk application.

Downloads

Download data is not yet available.

References

[1] L. S. Gunawan and C. S. T. Kansil, “Dampak Risiko Bagi Konsumen dalam Praktik Merugikan Pinjaman Online Ilegal,” J. Manaj. Pendidik. Dan Ilmu Sos., vol. 6, no. 1, pp. 461–469, 2024.

[2] A. P. Setiawan, D. Christensen, N. Ramadhani, S. M. Ridwan, and S. N. Azizah, “Fenomena Pinjaman Online sebagai Bentuk Kenakalan Terselubung di Kalangan Generasi Z: Studi Perspektif Sosial dan Kaitannya dengan Tujuan Pembangunan Berkelanjutan (SDGs),” J. Humaniora, Ekon. Syariah dan Muamalah, vol. 3, no. 2, pp. 23–29, 2025, doi: 10.38035/jhesm.v3i2.323.

[3] A. R. Putra and D. E. Ratnawati, “Analisis Sentimen Berbasis Aspek pada Aplikasi Mobile Menggunakan Naïve Bayes berdasarkan Ulasan Pengguna Playstore (Studi Kasus : Jconnect Mobile),” J. Teknol. Inf. dan Ilmu Komput., vol. 12, no. 2, pp. 293–300, 2025, doi: 10.25126/jtiik.2025127556.

[4] F. A. Irawan, A. R. Atmadja, and A. Wahana, “Analisis Sentimen Ulasan Aplikasi Bank Digital Menggunakan Algoritma Naïve Bayes,” J. Comput. Sci. Inf. Technol., vol. 4, no. 2, pp. 60–68, 2024, doi: 10.47065/explorer.v4i2.1181.

[5] D. Munandar, M. Afdal, Z. Zarnelly, and R. Novita, “Analisis Sentimen Ulasan Pengguna Aplikasi Mobile Banking Menggunakan Algoritma K-Nearest Neighbor,” J. Teknol. Sist. Inf. dan Apl., vol. 7, no. 3, pp. 1309–1318, 2024, doi: 10.32493/jtsi.v7i3.41409.

[6] K. Miranda and R. R. Suryono, “Analisis Sentimen Pinjaman Online: Studi Komparatif Algoritma Naïve Bayes, Decision Tree, dan KNN,” J. Pendidik. Inform., vol. 9, no. 2, pp. 372–381, 2025, doi: 10.29408/edumatic.v9i2.30142.

[7] S. Ainun Mardiah Hasibuan, “Klasifikasi Kelayakan Peminjaman Nasabah Menggunakan Algoritma K-Nearest Neighbor (K-NN),” J. Mach. Learn. Comput. Sci. J., vol. 4, no. 10, pp. 1525–1532, 2024.

[8] A. W. Septiano Anggun Pratama, “Perbandingan Klasifikasi Data Mining Untuk Prediksi Kredit Macet Dengan Menggunakan Metode Naïve Bayes Dan K – Nearest Neighbors (Studi Kasus: Bank BUMN Di Daerah Jawa Barat),” Simetris J. Tek. Mesin, Elektro dan Ilmu Komput., vol. 15, no. 2, pp. 1–13, 2024, doi: 10.24176/simet.v15i2.12321.

[9] S. R. Ardhana, T. Widiharih, and and B. A. Saputra, “Klasifikasi Menggunakan Algoritma K-Nearest Neighbor pada Imbalance Class Data dengan SMOTE. (Studi Kasus: Nasabah Bank Perkreditan Rakyat ‘X’),” Indones. J. Appl. Stat., vol. 6, no. 2, p. 152, 2024, doi: 10.13057/ijas.v6i2.79389.

[10] T. R. D. Putri1, M. R. Sanjaya, M. A. Firdaus, and D. R. Indah, “Implementasi Algoritma K-Nearest Neighbors dalam Analisis Sentimen Ulasan Aplikasi Bank Aladin,” J. Mach. Learn. Comput. Sci., vol. 6, no. April, pp. 608–618, 2026.

[11] K. Mustaqim and I. N. D. Fatia Amalia Maresti, “Analisis Sentimen Ulasan Aplikasi PosPay untuk Meningkatkan Kepuasan Pengguna dengan Metode K-Nearest Neighbor (KNN),” Edumatic J. Pendidik. Inform., vol. 8, no. 1, pp. 11–20, 2024, doi: 10.29408/edumatic.v8i1.24779.

[12] R. N. Muhammad Iqbal, Afdal, “Implementasi Algoritma Support Vector Machine Untuk Analisa Sentimen Data Ulasan Aplikasi Pinjaman Online di Google Play Store,” Indones. J. Mach. Learn. Comput. Sci., vol. 4, no. 4, pp. 1244–1252, 2024, doi: 10.57152/malcom.v4i4.1435.

[13] M. Rival Afandi, M Afdal, Rice Novita and Fakultas, “Analisis Sentimen Masyarakat Terhadap Pinjaman Online di Twitter Menggunakan Algoritma Naive Bayes Classifier dan K-Nearest Neighbor,” Build. Informatics, Technol. Sci., vol. 6, no. 2, pp. 596–605, 2024, doi: 10.47065/bits.v6i2.5300.

[14] S. Rifki Fahrezi Putra Ichsansyah, “Perbandingan Algoritma Naive Bayes Dan Random Forest Dalam Klasifikasi Sentimen Ulasan Pengguna Kredivo Di Play Store,” J. Teknol. Inf. dan Ilmu Komput., vol. 13, no. 2, pp. 297–308, 2026.

[15] M. D. Firmansyah, R. R. Setiawan, and Y. Irawan, “Analisis Perbandingan Kinerja Algoritma SVM, Naive Bayes Dan KNN Dalam Klasifikasi Sentimen Ulasan Aplikasi Pinterest Dengan SMOTE Dan PSO,” J. Teknol. dan Sist. Inf. Univrab, vol. 11, no. 1, pp. 639–652, 2026.

[16] I. M. T. Dian Tri Wilujeng , Mohamat Fatekurohman, “Analisis Risiko Kredit Perbankan Menggunakan Algoritma K-Nearest Neighbor dan Nearest Weighted K-Nearest Neighbor,” Indones. J. Appl. Stat., vol. 5, no. 2, pp. 142–148, 2023.

[17] K. Nika Nur Cahayana, Ulfi Saidata Aesyi, “Model Analisis Sentimen pada Ulasan Pengguna Mobile Banking Menggunakan Kombinasi K-Means dan Naive Bayes,” Angkasa J. Ilm. Bid. Teknol., vol. 17, no. 1, p. 44, 2025, doi: 10.28989/angkasa.v17i1.2500.

[18] N. Hafid and L. Khikmah, “Perbandingan Metode Algoritma C4 . 5 , Naïve Bayes , dan Logistic Regression untuk Penentuan Kelayakan Penerima Kredit Comparison of C4 . 5 , Naïve Bayes , and Logistic Regression Algorithm Methods for Determining Credit Recipient Eligibility,” Teknol. J. Ilm. Teknol. Sist. Inf., vol. 14, no. 2, pp. 85–93, 2024.