Comparison of Random Forest and Support Vector Machine Methods in Sentiment Analysis of Student Satisfaction Questionnaire Comments at ITB STIKOM Bali
DOI:
https://doi.org/10.30871/jaic.v9i3.9617Keywords:
Classification, Sentiment Analysis, Random Forest, Support Vector Machine, SMOTEAbstract
ITB STIKOM Bali is one of the higher education institutions in Bali that focuses on academic activities, particularly in the field of Information Technology. To maintain its educational quality, the Quality Assurance Department collaborates with the Center for Information and Communication (Puskom) to distribute a student satisfaction questionnaire at the end of each semester. In evaluating student satisfaction with campus facilities, the comment section is one of the key indicators, featuring the question: “Based on your experience, please describe which AAK services you found disappointing and in need of improvement.” This study compares the performance of the Random Forest and Support Vector Machine (SVM) methods in conducting sentiment analysis on historical student satisfaction comments. The research involved several stages, including literature review, data collection, preprocessing, transformation, data mining, evaluation, and visualization. The results demonstrate strong accuracy, precision, recall, and F1-scores for both methods using an 80:20 data split. Before applying the SMOTE technique, the best result was achieved by the Support Vector Machine method with a score of 0.90, while the Random Forest method yielded an accuracy of 0.81, precision of 0.85, recall of 0.81, and F1-score of 0.76. After applying SMOTE, both methods achieved an improved and equal score of 0.90. The study also produced an excellent classification result based on the ROC curve. It is expected that this research can serve as an additional reference for the assessment of student satisfaction at ITB STIKOM Bali at the end of each academic semester.
Downloads
References
[1] Ni Made Rai Masita Dewi, I Made Candiasa, and Kadek Yota Ernanda Aryanto, “Pengukuran Tingkat Kapabilitas Tata Kelola SION menggunakan Framework COBIT 5 pada Institut Teknologi dan Bisnis STIKOM Bali,” Jurnal Sistem dan Informatika (JSI), vol. 15, no. 2, pp. 144–154, May 2021, doi: 10.30864/jsi.v15i2.365.
[2] Septiana and Juhaini Alie, “Pengaruh Tangible, Reliability, Responsiveness, Assurance, dan EmphatyTerhadap Kepuasan Pelanggan Perempuan Pada Osbond GymPalembang,” Journal Management, Business, and Accounting, vol. 22, no. 2, pp. 325–339, Aug. 2023, doi: https://doi.org/10.33557/mbia.v22i2.2620.
[3] P. Kumala Sari and R. Randy Suryono, “Komparasi Algoritma Support Vector Machine Dan Random Forest Untuk Analisis Sentimen Metaverse,” Jurnal MNEMONIC, vol. 7, no. 1, pp. 31–39, Feb. 2024.
[4] B. Liu, “Sentiment Analysis: A Fascinating Problem,” in Sentiment Analysis and Opinion Mining, Cham: Springer International Publishing, 2012, pp. 1–8. doi: 10.1007/978-3-031-02145-9_1.
[5] I. Bagus Nyoman Wijana Manuaba, G. Rasben Dantes, and G. Indrawan, “Analisis Sentimen Data Provider Layanan Internet Pada Twitter Menggunakan Support Vector Machine (SVM) Dengan Penambahan Algoritma Levenshtein Distance,” Jurnal Sistem Komputer dan Kecerdasan Buatan, vol. 5, Mar. 2022.
[6] K. W. Trisna and H. J. Jie, “Deep Learning Approach for Aspect-Based Sentiment Classification: A Comparative Review,” Applied Artificial Intelligence, vol. 36, no. 1, 2022, doi: 10.1080/08839514.2021.2014186.
[7] N. M. K. Sedana, I. N. S. Wijaya, and I. K. R. Artana, “Analisis Sentimen Berbahasa Inggris Dengan Metode Lstm Studi Kasus Berita Online Pariwisata Bali English Sentiment Analysis Using The Lstm Method Case Study Of Bali Tourism Online News,” Jurnal Teknologi Informasi dan Ilmu Komputer (JTIIK), no. 6, pp. 1325–1334, 2024, doi: 10.25126/jtiik.2024118792.
[8] U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, “The KDD process for extracting useful knowledge from volumes of data,” Commun ACM, vol. 39, no. 11, pp. 27–34, 1996.
[9] I. P. Dedy, W. Darmawan, G. Aditra Pradnyana, I. Bagus, and N. Pascima, “Optimasi Parameter Support Vector Machine Dengan Algoritma Genetika Untuk Analisis Sentimen Pada Media Sosial Instagram,” SINTECH JOURNAL, vol. 6, Apr. 2023, [Online]. Available: https://doi.org/10.31598
[10] J. W. G. Putra, “Pengenalan konsep pembelajaran mesin dan deep learning,” Tokyo. Jepang, 2019.
[11] I. K. J. Arta, G. Indrawan, and G. R. Dantes, “Data Mining Rekomendasi Calon Mahasiswa Berprestasi Di Stmik Denpasar Menggunakan Metode Technique For Others Reference By Similarity To Ideal Solution,” Jurnal Sains & Teknologi, vol. 5, no. 2, pp. 2303–3142, Oct. 2016.
[12] L. Hermawan and M. B. Ismiati, “Pembelajaran Text Preprocessing berbasis Simulator Untuk Mata Kuliah Information Retrieval,” TRANSFORMATIKA, vol. 17, no. 2, pp. 188–199, Jan. 2020, doi: 10.26623/transformatika.v17i2.1705.
[13] N. M. A. J. Astari, Dewa Gede Hendra Divayana, and Gede Indrawan, “Analisis Sentimen Dokumen Twitter Mengenai Dampak Virus Corona Menggunakan Metode Naive Bayes Classifier,” Jurnal Sistem dan Informatika (JSI), vol. 15, no. 1, pp. 27–29, Nov. 2020, doi: 10.30864/jsi.v15i1.332.
[14] C. Fan, M. Chen, X. Wang, J. Wang, and B. Huang, “A Review on Data Preprocessing Techniques Toward Efficient and Reliable Knowledge Discovery From Building Operational Data,” Mar. 29, 2021, Frontiers Media S.A. doi: 10.3389/fenrg.2021.652801.
[15] A. Sasmita, G. A. Pradnyana, and D. G. H. Divayana, “Pengembangan Sistem Analisis Sentimen Untuk Evaluasi Kinerja Dosen Universitas Pendidikan Ganesha Dengan Metode Naïve Bayes,” JST (Jurnal Sains dan Teknologi), vol. 11, no. 2, pp. 451–462, Sep. 2022, doi: 10.23887/jstundiksha.v11i2.44384.
[16] H. D. Abubakar and M. Umar, “Sentiment Classification: Review of Text Vectorization Methods: Bag of Words, Tf-Idf, Word2vec and Doc2vec,” SLU Journal of Science and Technology, vol. 4, no. 1 & 2, pp. 27–33, Aug. 2022, doi: 10.56471/slujst.v4i.266.
[17] D. E. Cahyani and I. Patasik, “Performance comparison of tf-idf and word2vec models for emotion text classification,” Bulletin of Electrical Engineering and Informatics, vol. 10, no. 5, pp. 2780–2788, Oct. 2021, doi: 10.11591/eei.v10i5.3157.
[18] Y. Widyaningsih, G. P. Arum, and K. Prawira, “Aplikasi K-Fold Cross Validation Dalam Penentuan Model Regresi Binomial Negatif Terbaik,” Barekeng: Jurnal Ilmu Matematika dan Terapan, vol. 15, no. 2, pp. 315–322, Jun. 2021, doi: 10.30598/barekengvol15iss2pp315-322.
[19] M. Kuhn and K. Johnson, Applied predictive modeling, vol. 26. Springer, 2013.
[20] N. V Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” 2002.
[21] C. Ayunda et al., “Analisis Komparasi Algoritma Machine Learning untuk Sentiment Analysis (Studi Kasus: Komentar YouTube ‘Kekerasan Seksual’),” Jurnal Informatika: Jurnal pengembangan IT (JPIT), vol. 7, no. 2, pp. 80–84, May 2022.
[22] L. Breiman, “Random Forests,” Mach Learn, vol. 45, no. 1, pp. 5–32, Oct. 2001, doi: 10.1023/A:1010933404324.
[23] C. Cortes and V. Vapnik, “Support-vector networks,” Mach Learn, vol. 20, no. 3, pp. 273–297, 1995, doi: 10.1007/BF00994018.
[24] M. I. Fikri, T. S. Sabrila, and Y. Azhar, “Perbandingan Metode Naïve Bayes dan Support Vector Machine pada Analisis Sentimen Twitter,” SMATIKA Jurnal, vol. 10, pp. 71–76, Dec. 2020.
[25] R. A. Raharjo, I. Made, G. Sunarya, D. Gede, and H. Divayana, “Perbandingan Metode Naïve Bayes Classifier Dan Support Vector Machine Pada Kasus Analisis Sentimen Terhadap Data Vaksin Covid-19 Di Twitter,” Jurnal Ilmiah Elektronika Dan Komputer, vol. 15, no. 2, pp. 456–464, Dec. 2022, [Online]. Available: http://journal.stekom.ac.id/index.php/elkompage456
[26] I. K. W. Dananjaya and I. G. A. A. D. Indradewi, “Perbandingan Metode Pembobotan TF-RF Dan TF-ABS Pada Kategorisasi Berita Di BDI Denpasar,” Sintech Journal, vol. 6, no. 1, Apr. 2023.
[27] N. K. T. A. Saputri, I. G. A. Gunadi, and I. M. G. Sunarya, “Analisis Sentimen Pelayanan Daring di Fakultas Teknik dan Kejuruan Universitas Pendidikan Ganesha Menggunakan Algoritma Naïve Bayes dan LSTM,” MALCOM: Indonesian Journal of Machine Learning and Computer Science, vol. 4, no. 3, pp. 1120–1129, Jul. 2024, doi: 10.57152/malcom.v4i3.1336.
[28] K. Pearson, On the theory of contingency and its relation to association and normal correlation, vol. 1. London, Dulau and Co., 1904.
[29] J. Setyanto and T. B. Sasongko, “Sentiment Analysis of Sirekap Application Users Using the Support Vector Machine Algorithm,” Journal of Applied Informatics and Computing (JAIC), vol. 8, no. 1, pp. 2548–6861, Jul. 2024, [Online]. Available: http://jurnal.polibatam.ac.id/index.php/JAIC
[30] T. Rosandy, “Perbandingan Metode Naive Bayes Classifier Dengan Metode Decision Tree (C4.5) Untuk Menganalisa Kelancaran Pembiayaan (Study Kasus : Kspps / Bmt Al-Fadhila),” Jurnal TIM Darmajaya, vol. 02, pp. 52–62, May 2016.
[31] J. N. Mandrekar, “Receiver Operating Characteristic Curve in Diagnostic Test Assessment,” Journal of Thoracic Oncology, vol. 5, no. 9, pp. 1315–1316, Sep. 2010, doi: https://doi.org/10.1097/JTO.0b013e3181ec173d.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Purnama Sidik, I Made Gede Sunarya, I Gede Aris Gunadi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
