Knowledge Discovery of AI Usage Dependency Patterns in Learning Activities Using Random Forest, XGBoost, Logistic Regression with SHAP-Based Interpretation
DOI:
https://doi.org/10.30871/jaic.v10i3.12745Keywords:
AI dependency, Machine Learning, XGBoost, SHAP, Knowledge DiscoveryAbstract
The increasing use of Artificial Intelligence (AI) in education has influenced various learning activities. However, excessive AI usage has the potential to create dependency patterns that may affect students’ learning independence and critical thinking abilities. This study aims to analyze patterns of AI usage dependency in learning activities using a machine learning approach and to interpret the factors influencing such dependency. The analysis was conducted using a publicly available dataset representing usage intensity, session duration, AI assistance level, repeated usage behavior, and students’ academic characteristics. The research stages consisted of data preprocessing, categorical variable encoding, feature engineering, the construction of the Knowledge Dependency Level variable, class imbalance handling using SMOTE, and model evaluation using Stratified 5-Fold Cross Validation. The dataset was divided into 80% training data and 20% testing data, then modeled using Logistic Regression, Random Forest, and XGBoost. The results showed that XGBoost achieved the best performance with an accuracy of 0.6845, precision of 0.7288, recall of 0.6845, F1-score of 0.7028, and an AUC value of 0.860, indicating better discrimination capability compared to Random Forest and Logistic Regression. To support the knowledge discovery process, an interpretative analysis using SHAP was conducted to identify the contribution of each feature to the classification results. The interpretation revealed that SatisfactionRating was the most dominant feature influencing the prediction of AI usage dependency levels, followed by FinalOutcome, while academic factors such as StudentLevel and Discipline contributed relatively less. These findings transform previously implicit AI usage behavior patterns into explicit knowledge.
Downloads
References
[1] E. Wahyudinarti, P. Andini Rachmatika, and R. Nurul Ain, “Meningkatkan Efektivitas Pembelajaran Mahasiswa Dengan AI,” JATI (Jurnal Mhs. Tek. Inform., vol. 9, no. 1, pp. 488–491, 2025, doi: 10.36040/jati.v9i1.12279.
[2] E. Suryokta, W. Taruklimbong, and H. Sihotang, “Peluang dan Tantangan Penggunaan AI (Artificial Intelligence) dalam Pembelajaran Kimia,” J. Pendidik. Tambusai, vol. 7, no. 3, pp. 26745–26757, 2023.
[3] P. Techasermwattanakul and P. Suwannatthachote, “The Application Of Generative Ai In Educational Research: A Systematic Literature Review,” J. Educ. Innov., vol. 27, no. 1, pp. 160–174, 2025, doi: 10.71185/jeiejournals.v27i1.278408.
[4] J. A. Firdaus, R. I. Ummah, R. R. Aprialini, A. Fithriyyah, Mahsus, and A. Faizin, “Ketergantungan Penggunaan Kecerdasan Buatan (AI) pada Tugas Akademik Mahasiswa Terhadap Kemampuan Berpikir Kritis dan Kreatif,” J. Kependidikan, vol. 14, no. 1, pp. 1203–1214, 2025.
[5] T. N. Mahmud et al., “Cognitive Offloading dalam Penggunaan Generative Artificial Intelligence (GAI) dan Perannya terhadap Working Memory Mahasiswa: Scoping Review,” J. Pendidik. MIPA, vol. 16, no. 2, pp. 488–500, 2026.
[6] Lukman, R. Agustina, and R. Aisy, “Problematika Penggunaan Artificial Intelligence untuk Pembelajaran di Kalangan Mmahasiswa,” J. Madaniyah, vol. 13, pp. 1–14, 2023.
[7] S. Marisa, D. Rachman, R. H. Puspita, and Y. Rahmawati, “The Dark Side of Dependency: Negative Consequences of EFL Students’ Use of ChatGPT for Academic Writing,” Borneo Educ. J., vol. 7, no. 2, pp. 255–274, 2025, doi: 10.24903/bej.v7i2.2069.
[8] Z. Revesai, “Generative AI dependency: the emerging academic crisis and its impact on student performance—a case study of a university in Zimbabwe,” Cogent Educ., vol. 12, no. 1, p., 2025, doi: 10.1080/2331186X.2025.2549787.
[9] Kurniahtunnisa, P. T. Agustina, M. Y. Manuel, M. Rukmana, and K. Lestari, “Analisis Penggunaan Artificial Intelligence (AI) dalam Pembelajaran IPA di Sekolah: Persepsi dan Preferensi Siswa SMA,” J. HOMEPAGE http//ejurnal.unima.ac.id/index.php/sciening Pengaruh, vol. Vol.3 No., no. 2, pp. 64–70, 2024, doi: 10.53682/slj.v5i2.10431.
[10] S. Rahmadani, D. Z. Rizky, and N. Aini, “Eksplorasi Kebiasaan Gen-Z dalam Menggunakan AI untuk Beraktivitas Sehari-Hari,” vol. 5, no. 3, pp. 327–339, 2025.
[11] M. Abbas, F. A. Jam, and T. I. Khan, “Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students,” Int. J. Educ. Technol. High. Educ., vol. 21, no. 1, 2024, doi: 10.1186/s41239-024-00444-7.
[12] S. Sallu and H. Hendriadi, “Random Forest Machine Learning Analysis of Generative AI’s Impact on Learning Effectiveness in Indonesian Higher Education,” J. Tek. Inform., vol. 6, no. 5, pp. 3635–3651, 2025, doi: 10.52436/1.jutif.2025.6.5.5365.
[13] T. Agustina, M. Masrizal, and I. Irmayanti, “Performance Analysis of Random Forest Algorithm for Network Anomaly Detection using Feature Selection,” Sinkron, vol. 8, no. 2, pp. 1116–1124, 2024, doi: 10.33395/sinkron.v8i2.13625.
[14] Florentina Tatrin Kurniati and Dian Pramana, “Identifikasi Objek Menggunakan Random Forest dan Multi-Fitur,” J. Sist. dan Inform., vol. 17, no. 2, pp. 130–136, 2023, doi: 10.30864/jsi.v17i2.590.
[15] G. Airlangga, “Application of Traditional Machine Learning Techniques for the Classification of Human DNA Sequences: A Comparative Study of Random Forest and XGBoost,” J. Inform. Univ. Pamulang, vol. 9, no. 1, pp. 23–30, 2024, doi: 10.32493/informatika.v9i1.39353.
[16] D. Amanda Ardhani and K. D. Tania, “Knowledge Discovery on E-Commerce Customer Churn Using Interpretable Machine Learning: A Comparative Study of SHAP-Based Classifiers,” J. Appl. Informatics Comput., vol. 9, no. 5, pp. 2695–2702, 2025, doi: 10.30871/jaic.v9i5.10811.
[17] R. Rismala, L. Novamizanti, K. N. Ramadhani, Y. S. Rohmah, S. Parjuangan, and D. Mahayana, “Kajian Ilmiah dan Deteksi Adiksi Internet dan Media Sosial di Indonesia Menggunakan XGBoost,” J. Edukasi dan Penelit. Inform., vol. 7, no. 1, p. 1, 2021, doi: 10.26418/jp.v7i1.43606.
[18] Christine Eirene, Dian Syafitri, Neny Sulistianingsih, Khasnur Hidjah, and Hairani Hairani, “Classification of Learning Styles of Junior High School Students Using Random Forest & XGBoost Algorithm,” J. Bumigora Inf. Technol., vol. 7, no. 1, pp. 15–24, 2025, doi: 10.30812/bite.v7i1.4913.
[19] F. I. Adani and H. Amalia, “Penerapan Metode Algoritma XGBoost untuk Prediksi Risiko Penyakit Jantung,” J. Insa. J. Inf. Syst. Manag. Innov., vol. 5, no. 2, pp. 117–125, 2025, doi: 10.31294/j-insan.v5i2.10000.
[20] M. A. R. Yudha and M. Rahardi, “Comparative Analysis of Random Forest and XGBoost Models for Cervical Cancer Risk Prediction using SHAP-based Explainable AI,” J. Appl. Informatics Comput., vol. 9, no. 6, pp. 3198–3211, 2025, doi: 10.30871/jaic.v9i6.10357.
[21] M. Safii, Husain, Ika Okta Kirana, Sasha Aiko Leana, and Yuli Indahwati Gultom, “Model Prediksi Penjadwalan Produksi Energi Terbarukan dengan Algoritma XGBoost dan Analisis Interpretatif Menggunakan SHAP,” J. Sist. Inf. Triguna Dharma (JURSI TGD), vol. 4, no. 4, pp. 794–801, 2025, doi: 10.53513/jursi.v4i4.11443.
[22] S. R. Ais and U. P. Sanjaya, “Perbandingan Algoritma Random Forest, XGBoost, dan Logistic Regression untuk Prediksi Risiko Kekambuhan Kanker Tiroid,” Edumatic J. Pendidik. Inform., vol. 9, no. 1, pp. 324–332, 2025, doi: 10.29408/edumatic.v9i1.29644.
[23] Jeffry, A. I. S. Azis, and E. T. J. Kandakon, “Integrating Bayesian Optimization into Ensemble Logistic Regression for Explainable AI-Based Customer Behavior Analysis,” Sink. J. dan Penelit. Tek. Inform., vol. 9, no. 4, pp. 1900–1911, 2025.
[24] M. Kumar, J. Wadhwa, and A. Qtaishat, “Utilizing Random Forest and XGBoost Data Mining Algorithms for Anticipating Students ’ Academic Performance,” vol. 16, no. 2, pp. 29–44, 2024, doi: 10.5815/ijmecs.2024.02.03.
[25] C. Adipradana, I. Taufiq, and A. N. Izzati, “Analisis Kelayakan Penerima Program Keluarga Harapan ( PKH ) di Kecamatan Pujud Kabupaten Rokan Hilir Menggunakan Logistic Regression,” vol. 1, pp. 77–86, 2026.
[26] M. Anshari, M. Syafrudin, A. Tan, and N. L. Fitriyani, “Optimisation of Knowledge Management ( KM ) with Machine Learning ( ML ) Enabled,” no. Ml, pp. 1–15, 2023.
[27] D. A. Shafiq, M. Marjani, R. Ahamed, A. Habeeb, and D. Asirvatham, “Student Retention Using Educational Data Mining and Predictive Analytics : A Systematic Literature Review,” IEEE Access, vol. 10, no. July, pp. 72480–72503, 2022, doi: 10.1109/ACCESS.2022.3188767.
[28] E. Kalita et al., “Educational data mining : a 10 ‑ year review,” Discov. Comput., 2025, doi: 10.1007/s10791-025-09589-z.
[29] A. Risyahputri, D. Kurniawan, and K. Ditha Tania, “Comparison of XGBoost and LSTM in Knowledge Discovery for GrokAI Mobile Application Sentiment Analysis,” Technol. Sci., vol. 7, no. 3, pp. 1637–1648, 2025, doi: 10.47065/bits.v7i3.8651.
[30] E. Hermawan, S. Darmawan Panjaitan, and E. Faja Ripanti, “Sistem Prediksi Banjir Rob Kota Pontianak Berbasis Machine Learning Menggunakan Framework Streamlit,” JEPIN (Jurnal Edukasi dan Penelit. Inform., vol. 10, no. 3, pp. 351–361, 2024.
[31] B. Flanagan, R. Majumdar, and H. Ogata, “Fine Grain Synthetic Educational Data : Challenges and Limitations of Collaborative Learning Analytics,” IEEE Access, vol. 10, pp. 26230–26241, 2022, doi: 10.1109/ACCESS.2022.3156073.
[32] A. Azrul, A. I. Purnamasari, and I. Ali, “Analisis Sentimen Pengguna Twitter Terhadap Perkembangan Artificial Intelligence Dengan Penerapan Algoritma Long Short-Term Memory ( LTSM ),” vol. 8, no. 1, pp. 413–421, 2024.
[33] A. Widianti and I. Pratama, “Penanganan Missing Values Dan Prediksi Data Timbunan,” vol. 9, no. 2, pp. 242–251, 2024.
[34] H. L. Wati, N. Anggraeni, S. Kolbiah, U. Hendar, and N. Agustina, “Perbandingan Algoritma Random Forest Dan Naratif : Jurnal Ilmiah Nasional Riset Aplikasi Dan Teknik Informatika,” vol. 07, no. 01, pp. 64–71, 2025.
[35] M. Irawan, R. N., Hindrayani, K. M., & Idhom, “Penerapan Cross Validationsebagai Analisis Sentimen Pelayanan Publik Kereta Api Lokal Daop 8 Menggunakan Metode Multinomial Naïve Bayes,” G-Tech J. Teknol. Terap., vol. 6, no. 2, pp. 295–305, 2022.
[36] P. P. Allorerung, A. Erna, M. Bagussahrir, and S. Alam, “Analisis Performa Normalisasi Data untuk Klasifikasi K-Nearest Neighbor pada Dataset Penyakit,” vol. 9, no. 3, pp. 178–191, 2024.
[37] C. Wastu et al., “Pengukuran Indeks Pembangunan Perkebunan Di Jawa Timur Dengan Pendekatan Indeks Komposit Dan Analisis Faktor Measurement Of Plantation Development Index In East Java,” vol. 23, no. 02, pp. 331–344, 2025.
[38] M. H. Bisri, N. A. Setianto, and M. Bata, “Analisis Potensi Sda Dan Sdm Usaha Kelompok Ternak Sapi Potong Sistem Semi Intensif Integrasi Sapi-Sawit Di Bangka Tengah : " Pendekatan Indeks Komposit ",” pp. 161–167, 2025.
[39] A. W. Jeremy, J. Kelleher, J. Magid, B. R. Jackson, M. E. Pennini, and D. Kushner, “Dispensing of Oral Antiviral Drugs for Treatment of COVID-19 by Zip Code – Level Social Vulnerability — United States, December 23, 2021–May 21, 2022 Jeremy,” vol. 71, no. 25, pp. 825–829, 2022.
[40] E. Barth, J. Tadeu, V. De Resende, and K. H. Mariguele, “Multivariate analysis methods improve the selection of strawberry genotypes with low cold requirement,” Sci. Rep., pp. 1–12, 2022, doi: 10.1038/s41598-022-15688-4.
[41] R. Z. Arifin, H. Firmansyah, and W. Asriyani, “Prediksi Kelulusan Siswa Berdasarkan Data Demografis dan Akademik pada Dataset Student Performance,” vol. 4, no. 2, pp. 13300–13307, 2025.
[42] B. A. C. Permana, Zulkipli, M. Wasil, and Harianto, “Improving Imbalanced Polycystic Ovary Syndrome Classification Using a Leakage-Free Machine Learning Pipeline,” vol. 9, no. January, pp. 538–547, 2026.
[43] A. Setiawan, R. Yanti, E. Ali, and H. Yenni, “Optimasi Klasifikasi Tingkat Obesitas Pada Remaja Berdasarkan Pola Hidup Menggunakan SVM Dengan Teknik Smote,” pp. 224–235, 2025, doi: 10.33364/algoritma/v.22-2.2509.
[44] Amiruddin and R. Ishak, “Implementasi Seleksi Fitur Klasifikasi Waktu Kelulusan Mahasiswa Menggunakan Correlation Matrix With Heatmap,” vol. 4, pp. 169–174, 2022.
[45] A. I. Pradana and V. Atina, “Teknik K-Fold Cross Validation untuk Mengevaluasi Kinerja Mahasiswa,” pp. 239–248, 2024, doi: 10.33364/algoritma/v.21-1.1618.
[46] C. Arafat et al., “Perbandingan Algoritma Random Forest Dan Xgboost Untuk Klasifikasi Penyakit Jantung Berdasarkan Data Medis,” vol. 15, no. 2, pp. 430–435, 2025.
[47] J. P. Anggraini, C. G. Z. A, and A. Desiani, “Komputika : Sistem Komputer Perbandingan Algoritma Random Forest dan Extreme Gradient Boosting ( XGBoost ) dalam Klasifikasi Penyakit Gagal Jantung Comparison of Random Forest and Extreme Gradient Boosting ( XGBoost ) Algorithms in Heart Failure Disease C,” vol. 12, no. 2, 2025, doi: 10.34010/komputika.v14i2.16618.
[48] M. N. Ramadhani, K. D. Tania, and M. Afrina, “Knowledge Discovery in Sharia Mobile Banking Reviews Using Aspect- Based Sentiment Analysis and Machine Learning,” vol. 10, no. 1, pp. 640–650, 2026.
[49] O. B. Limet et al., “Pemilihan Fitur Untuk Prediksi Kematangan Kelapa Sawit Menggunakan Explainable AI ( XAI ) Feature Selection for Oil Palm Ripeness Prediction Using Explainable AI,” vol. 7, no. 2, pp. 137–150, 2025.
[50] L. N. Hapsari, I. Fannani, Y. Rahmawati, and A. Muhariya, “Explainable Machine Learning untuk Prediksi Risiko Penyakit Jantung Menggunakan Random Forest dan Analisis SHAP,” REMIK Ris. dan E-Jurnal Manaj. Inform. Komput., vol. 10, no. 1, pp. 190–199, 2026, doi: http://doi.org/10.33395/remik.v10i1.15766.
[51] N. Ardana, H. Indrawati, and F. Trisnawati, “Pengaruh Pemanfaatan Teknologi Artificial Intelligence terhadap Kemandirian Belajar ( Studi pada Mahasiswa Jurusan Pendidikan IPS Universitas Riau ),” JIIP (Jurnal Ilm. Ilmu Pendidikan), vol. 8, no. 8, pp. 9688–9699, 2025.
[52] U. Hanifah and N. Novebri, “Ketergantungan Penggunaan Aplikasi AI dalam Keefektivitasan Belajar pada Mahasiswa Manajemen Pendidikan Islam,” J. Manaj. dan Pendidik. Agama Islam, vol. 3, no. 1, pp. 265–273, 2024, doi: 10.61132/jmpai.v3i1.866.
[53] Kurniahtunnisa, M. Y. Manuel, M. Aini, and T. P. Agustina, “Persepsi dan Sikap Siswa Terhadap Penggunaan Artificial Intelligence,” Sch. J. Pendidik. dan Kebud., vol. 15, no. 1, pp. 47–59, 2025.
[54] L. K. Dewi and N. I. Lahizha, “Integrasi Artificial Intelligence ( AI ) dalam Sistem Pembelajaran Adaptif untuk Meningkatkan Belajar Mandiri Mahasiswa,” JIIP (Jurnal Ilm. Ilmu Pendidikan), vol. 8, no. September, pp. 10916–10921, 2025.
[55] M. A. Failandri, M. Fatkhunihad, L. Shodik, M. R. Rizqiyanto, and D. A. Nugroho, “Analisis Tingkat Ketergantungan Mahasiswa Informatika Terhadap Penggunaan AI,” J. REKAYASA Inf. SWADHARMA, vol. 06, no. 1, 2026.
[56] Nisma and F. I. Lisanty, “Hubungan Intensitas Penggunaan Aplikasi AI Dengan Efektivitas Belajar Pada Mahasiswa Kebidanan,” ILLEA J. Heal. Sci. Public Heal. Med., vol. 1, no. 2, 2025.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Fidela Tertia Alfino, Puti Chalisa Wardhana, A. Salwa Aurelya Putri, Athiyyah Nuha Rotifa, Ken Ditha Tania, Ahmad Rifai, Dedy Kurniawan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
