TikTok Sentiment Analysis on Koperasi Merah Putih Using SVM and ANN

Meliysa Pasa Bagna Aprilia Said; Kholifatus Sholihah; Ifnu Wisma Dwi Prastya; Afril Efan Pajri

doi:10.30871/jaic.v10i4.13394

Authors

Meliysa Pasa Bagna Aprilia Said Universitas Nahdlatul Ulama Sunan Giri
Kholifatus Sholihah Universitas Nahdlatul Ulama Sunan Giri
Ifnu Wisma Dwi Prastya Universitas Nahdlatul Ulama Sunan Giri
Afril Efan Pajri Universitas Nahdlatul Ulama Sunan Giri

DOI:

https://doi.org/10.30871/jaic.v10i4.13394

Keywords:

ANN, Koperasi Merah Putih, Lexicon-Based, SVM, TikTok

Abstract

TikTok has become a relevant social media source for observing public responses to public issues, including the Koperasi Merah Putih program. This study compares Support Vector Machine (SVM) and Artificial Neural Network (ANN) for classifying sentiment in TikTok comments. The dataset was obtained through TikTok comment scraping and consisted of 25,669 raw comments. After removing empty comments and applying preprocessing stages consisting of cleaning, case folding, normalization, tokenization, stopword removal, and stemming, 21,026 comments were used for sentiment analysis. Sentiment labels were generated automatically using a lexicon-based sentiment labeling approach and grouped into three classes: positive, negative, and neutral. TF-IDF was used for feature extraction with a maximum of 5,000 features and unigram-bigram representation. The dataset was split into training and testing sets with an 80:20 ratio, while Stratified K-Fold Cross Validation and SMOTE were applied to strengthen evaluation and address class imbalance. The results show that SVM achieved the best overall performance before SMOTE with an accuracy of 86.66% and an F1-score of 86.76%. ANN achieved an accuracy of 85.31% before SMOTE and improved slightly after SMOTE to 85.47%. These findings indicate that SVM is more stable for TF-IDF-based TikTok comment classification, while SMOTE can improve ANN performance slightly but does not always increase all models equally.

Downloads

Download data is not yet available.

References

[1] R. E. Nurfirdaus, M. A. Barata, and I. W. Prastya, “Comparison of SVM and Random Forest for TikTok E10 Fuel Sentiment Analysis,” J. Appl. Informatics Comput., vol. 10, no. 2, pp. 1426–1437, 2026, [Online]. Available: https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/12395

[2] R. L. Mulianingrum, “Comparative Performance of SVM and BERT-Base Using TikTok Comments for Sentiment Analysis,” J. Appl. Informatics Comput., 2025, [Online]. Available: https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/11385

[3] S. R. Putri, A. Asrianda, and L. Rosnita, “Sentiment Analysis of Youtube and Gotube Reviews on Google Play Using the Support Vector Machine (SVM) Method in Indonesia,” J. Appl. Informatics Comput., vol. 9, no. 3, pp. 1025–1033, 2025, doi: 10.30871/jaic.v9i3.9461.

[4] B. R. Ermawan, “Optimizing Sentiment Classification Models for TikTok Comments Using Emotion-Based Preprocessing and Hyperparameter Optimization,” J. Appl. Informatics Comput., 2026, [Online]. Available: https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/11742

[5] A. P. Widyasari and Muljono, “Sentiment Analysis of YouTube Comments on the 2025 DPR RI Demonstration Using Machine Learning,” J. Appl. Informatics Comput., 2025, [Online]. Available: https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/12254

[6] J. Pardede, “Perbandingan Algoritma Stemming Porter, Sastrawi, Idris, dan Enhanced Confix Stripping pada Information Retrieval,” J. Teknol. Inf. dan Ilmu Komput., 2025, [Online]. Available: https://jtiik.ub.ac.id/index.php/jtiik/article/view/8860

[7] S. Febriani, V. Wati, Y. Wijayanti, and I. Siswanto, “Twitter Sentiment Analysis on Digital Payment in Indonesia Using Artificial Neural Network,” J. Appl. Informatics Comput., vol. 9, no. 2, 2025, [Online]. Available: https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/8988

[8] A. Widodo, “Optimizing Support Vector Machine (SVM) for Sentiment Analysis Using TF-IDF, SMOTE, and Chi-Square,” J. Appl. Informatics Comput., 2025, [Online]. Available: https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/10541

[9] M. Ilmiyah, M. A. Barata, and P. E. Yuwita, “Implementation of ANN Optimization with SMOTE and Backward Elimination for PCOS Prediction,” Sci. J. Informatics, 2025, [Online]. Available: https://scholar.google.com/citations?hl=en&user=iyoAaIEAAAAJ

[10] A. L. D. Ulhaq and S. Suprayogi, “Comparing Machine Learning Models for Sentiment Analysis of Tokopedia Reviews,” J. Appl. Informatics Comput., vol. 9, no. 6, 2025, [Online]. Available: https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/11239

[11] H. Barus, I. N. Fajri, and Y. Pristyanto, “Sentiment Classification Analysis of Tokopedia Reviews Using TF-IDF, SMOTE, and Traditional Machine Learning Models,” J. Appl. Informatics Comput., vol. 9, no. 5, 2025.

[12] F. Koto and G. Y. Rahmaningtyas, “InSet Lexicon: Evaluation of a Word List for Indonesian Sentiment Analysis in Microblogs,” in in Proc. 21st International Conference on Asian Language Processing (IALP), 2017. [Online]. Available: https://github.com/fajri91/InSet

[13] N. V Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” J. Artif. Intell. Res., vol. 16, pp. 321–357, 2002, [Online]. Available: https://arxiv.org/abs/1106.1813

[14] D. Musfiroh, “Sentiment Analysis of Online Lectures in Indonesia from Twitter Dataset Using InSet Lexicon,” MALCOM Indones. J. Mach. Learn. Comput. Sci., 2021, [Online]. Available: https://journal.irpi.or.id/index.php/malcom/article/download/20/11

[15] M. Mujahid, “Data oversampling and imbalanced datasets: an investigation of performance for machine learning and feature engineering,” J. Big Data, vol. 11, no. 87, 2024, doi: 10.1186/s40537-024-00943-4.

[16] R. T. Aldisa and P. Maulana, “Analisis Sentimen Opini Masyarakat Terhadap Vaksinasi Booster COVID-19 Dengan Perbandingan Metode Naive Bayes, Decision Tree dan SVM,” Build. Informatics, Technol. Sci., vol. 4, no. 1, pp. 106–109, 2022, doi: 10.47065/bits.v4i1.1581.

[17] F. Ansyah and R. R. Suryono, “Sentiment Classification of Indonesian-Language Roblox Reviews Using IndoBERT with SMOTE Optimization,” J. Appl. Informatics Comput., vol. 9, no. 4, 2025.