Enhancing Cyberbullying Sentiment Detection: A Comparative Study of IndoBERT and IndoBERTweet over SMOTE and Bernoulli Naive Bayes Approach
DOI:
https://doi.org/10.30871/jaic.v10i3.12632Keywords:
BERT, Cyberbullying, IndoBERT, IndoBERTweet, Natural Language Processing (NLP), Sentiment AnalysisAbstract
Cyberbullying has become a critical issue in social media use because it can negatively impact users’ mental health and social interactions. The high volume of aggressive comments and hate speech on digital platforms highlights the need for an automatic detection system that can accurately and reliably identify cyberbullying content. This research compares the performance of Indonesian language transformer models, IndoBERT and IndoBERTweet, in detecting text-based cyberbullying. Before modeling, the dataset undergoes Exploratory Data Analysis to understand its characteristics, class distribution, comment length, and potential data imbalance. Next, text preprocessing and tokenization are performed before dividing the data using stratified holdout splitting to preserve class proportions in training and testing sets. Both models are then trained with the same hyperparameter settings to ensure an objective and fair performance comparison. Results show that IndoBERT achieved an accuracy of 0.8333, while IndoBERTweet performed better with an accuracy of 0.8409. The analysis of the confusion matrix and ROC curve confirms that IndoBERTweet is more effective at detecting cyberbullying across different classes. Compared to previous studies using the SMOTE method and Bernoulli Naïve Bayes algorithm, which achieved 84.00% accuracy, this study's findings are slightly higher at 84.09%. Notably, this was achieved without using synthetic oversampling techniques. This suggests that the approach employed in this research can deliver competitive performance even without data balancing with SMOTE. Overall, these findings indicate that a transformer-based approach, combined with a more representative dataset, can improve cyberbullying detection more efficiently and practically. Therefore, IndoBERTweet is a more suitable model for implementing a cyberbullying content moderation system in Indonesia.
Downloads
References
[1] G. Ray, C. D. McDermott, and M. Nicho, “Cyberbullying on Social Media: Definitions, Prevalence, and Impact Challenges,” Journal of Cybersecurity, vol. 10, no. 1. Oxford University Press, 2024. doi: 10.1093/cybsec/tyae026.
[2] C. Murphy, “Cyberbullying among young people: Laws and policies in selected Member States,” EPRS | Eur. Parliam. Res. Serv., no. June 2024, pp. 1–12, 2024.
[3] N. Fiddiana and A. Bagus Priyambodo, “The Correlation Between Self-Control and Cyberbullying at Private High School X in Bogor,” KnE Soc. Sci., vol. 2021, no. ICoPsy 2021, pp. 255–266, 2022, doi: 10.18502/kss.v7i1.10216.
[4] D. Herlambang, R. Zildjianda, and D. Brajannoto, “Analysis of Cyberbullying Among Students: a Legal Perspective in Indonesia,” Entita J. Pendidik. Ilmu Pengetah. Sos. dan Ilmu-Ilmu Sos., no. 11, pp. 51–62, 2025, doi: 10.19105/ejpis.v1i.19134.
[5] N. Agustiningsih, A. Yusuf, A. Ahsan, and Q. Fanani, “The impact of bullying and cyberbullying on mental health: a systematic review,” Int. J. Public Heal. Sci., vol. 13, no. 2, p. 513, Jun. 2024, doi: 10.11591/ijphs.v13i2.23683.
[6] G. J. Hjetland et al., “Digital self ‑ presentation and adolescent mental health : Cross ‑ sectional and longitudinal insights from the ‘ LifeOnSoMe ’‑ study,” BMC Public Health, 2024, doi: 10.1186/s12889-024-20052-4.
[7] K. Fitzgerald, L. Vandenbosch, and T. Tabruyn, “Adolescent Emotional Expression on Social Media: A Data Donation Study Across Three European Countries,” Affect. Sci., vol. 5, no. 4, pp. 436–448, Dec. 2024, doi: 10.1007/s42761-024-00259-9.
[8] N. Aulia Maharani and W. Wahyunengsih, “Social Media Instagram as a News Reference for Journalism Students,” J. Karya Abdi Masy., vol. 8, no. 1, pp. 41–48, 2024, doi: 10.22437/jkam.v8i1.26176.
[9] L. Ismail, U. Mukramin, A. Nursida, and U. Muhammadiyah Makassar, “The Impact of Hate Comments on Social Media Users’ Self-Confidence: An Analysis Grounded in Social Comparison Theory and Cyberbullying Research,” PELNUS | Pen en Light Nat. Union Sci., vol. 3, no. 2, pp. 49–59, 2024, doi: https://doi.org/10.35335/psychologia.v3i2.57.
[10] M. Couto, A. Perez, J. Parapar, and D. E. Losada, “Temporal Word Embeddings for Early Detection of Psychological Disorders on Social Media,” J. Healthc. Informatics Res., 2025, doi: 10.1007/s41666-025-00186-9.
[11] F. Ramadan, E. Hassan, and F. A. Omara, “Cyberbullying detection in social media using natural language processing,” Sci. African, vol. 28, no. April, p. e02713, 2025, doi: 10.1016/j.sciaf.2025.e02713.
[12] A. S. Talaat, “Sentiment analysis classification system using hybrid BERT models,” J. Big Data, 2023, doi: 10.1186/s40537-023-00781-w.
[13] “Matematika Pada Kecerdasan Buatan - Widyastuti Andriyani, Mochammad Anshori, Dwi Normawati, Risqy Siwi Pradini, Mohamad Zaenudin, Muhammad Iqbal Harisuddin, M. Syauqi Haris, Astuty, Anna Angela Sitinjak, Wahyu Teja Kusuma - Google Books.” Accessed: Feb. 26, 2026. [Online]. Available: https://books.google.co.id/books?hl=en&lr=&id=cEYhEQAAQBAJ&oi=fnd&pg=PP1&ots=q4w9L__NSd&sig=p6pu2rV06VpfrZfkh6oJometj6k&redir_esc=y#v=onepage&q&f=false
[14] M. L. Jamil, S. Pais, J. Cordeiro, and G. Dias, “Detection of extreme sentiments on social networks with BERT,” Soc. Netw. Anal. Min., vol. 12, no. 1, pp. 1–16, Dec. 2022, doi: 10.1007/s13278-022-00882-z.
[15] U. States, “Deep Learning--based Text Classification: A Comprehensive Review,” ACM Comput. Surv., vol. 54, no. 3, 2026, doi: 10.1145/3439726.
[16] N. E. Zekaoui, S. Yousfi, M. Rhanoui, and M. Mikram, “Analysis of the evolution of advanced transformer-based language models : experiments on opinion mining,” IAES Int. J. Artif. Intell., vol. 12, no. 4, 2023.
[17] N. Mushtaq, G. Ali, D. Muhammad, K. Malik, and A. Bukhari, “BERT applications in natural language processing : a review,” Springer Nat., vol. 58, 2025, [Online]. Available: https://doi.org/10.1007/s10462-025-11162-5
[18] P. Xue and W. Bai, “A Fine-Grained Sentiment Analysis Method Using Transformer for Weibo Comment Text,” Int. J. Inf. Technol. Syst. Approach, vol. 17, no. 1, pp. 1–24, Jul. 2024, doi: 10.4018/ijitsa.345397.
[19] A. Amrullah, “Sentiment Analysis in the Age of Transformers and Large Language Models : A Comprehensive Review of Recent Advances and Future,” Intellithings J., vol. 1, no. 1, 2025.
[20] K. Kamdan, M. P. Anugrah, M. J. Almutaali, R. Ramdani, and I. L. Kharisma, “Performance Analysis of IndoBERT for Detection of Online Gambling Promotion in YouTube Comments †,” Eng. Proc., vol. 107, no. 1, 2025, doi: 10.3390/engproc2025107066.
[21] A. A. Hafiza and E. B. Setiawan, “Enhancing Cyberbullying Detection on Platform ‘X’ Using IndoBERT and Hybrid CNN-LSTM Model,” J. Tek. Inform., vol. 6, no. 2, pp. 655–672, Apr. 2025, doi: 10.52436/1.jutif.2025.6.2.4321.
[22] M. S. M. Faza, U. Islam, S. Agung, and A. Azka, “Penerapan Model IndoBERT untuk Deteksi Potensi Sumber Stres dalam Teks Media Sosial,” J. Rekayasa Sist. danTeknologi, vol. 3, no. 1, pp. 271–283, 2025, doi: https://doi.org/10.70248/jrsit.v3i1.3046.
[23] M. Fadhel and W. Maharani, “Depression Detection of Users in Social Media X using IndoBERTweet,” Sinkron, vol. 8, no. 2, pp. 885–891, 2024, doi: 10.33395/sinkron.v9i2.13354.
[24] Akira Permata Ramadhani, Eka Dyar Wahyuni, and Amalia Anjani Arifiyanti, “Klasifikasi Cyberbullying pada Komentar Instagram dengan Menggunakan Supervised Learning,” Neptunus J. Ilmu Komput. Dan Teknol. Inf., vol. 2, no. 2, pp. 92–101, 2024, doi: 10.61132/neptunus.v2i2.108.
[25] I. R. Hidayat and W. Maharani, “General Depression Detection Analysis Using IndoBERT Method,” Int. J. Inf. Commun. Technol., vol. 8, no. 1, pp. 41–51, 2022, doi: 10.21108/ijoict.v8i1.634.
[26] F. K. J. H. L. T. Baldwin, “Indobertweet : A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization,” Assoc. Comput. Linguist., vol. https://ac, pp. 10660–10668, 2021, doi: https://doi.org/10.18653/v1/2021.emnlp-main.833.
[27] E. D. Wahyuni, “Indonesia Instagram cyberbullying,” Mendeley Data. [Online]. Available: https://data.mendeley.com/datasets/xthb26ntc5/1
[28] I. O. Muraina, O. M. Adesanya, M. A. Agoi, and S. Onen, “The Necessity of Exploratory Data Analysis : How are preprocessing activities beneficial to Data Analysts and Professional Researchers in Academia ?,” Int. J. Sci. Res. Comput. Sci. Eng., vol. 11, no. 3, pp. 22–28, 2023.
[29] H. T. Duong and T. A. Nguyen-Thi, “A review: preprocessing techniques and data augmentation for sentiment analysis,” Comput. Soc. Networks, vol. 8, no. 1, pp. 1–16, Dec. 2021, doi: 10.1186/s40649-020-00080-x.
[30] M. Kastrati, Z. Kastrati, A. Shariq Imran, and M. Biba, “Leveraging distant supervision and deep learning for twitter sentiment and emotion classification,” J. Intell. Inf. Syst., vol. 62, no. 4, pp. 1045–1070, Aug. 2024, doi: 10.1007/s10844-024-00845-0.
[31] M. Siino, I. Tinnirello, and M. La Cascia, “Is text preprocessing still worth the time ? A comparative survey on the influence of popular preprocessing methods on Transformers and traditional classifiers,” Inf. Syst., vol. 121, no. July 2023, p. 102342, 2024, doi: 10.1016/j.is.2023.102342.
[32] N. A. Salsabila, Y. Ardhito, W. Ali, A. Septiandri, and A. Jamal, “Colloquial Indonesian Lexicon,” 2018 Int. Conf. Asian Lang. Process., pp. 226–229, 2018.
[33] C. W. Schmidt et al., “Tokenization Is More Than Compression,” EMNLP 2024 - 2024 Conf. Empir. Methods Nat. Lang. Process. Proc. Conf., pp. 678–702, 2024, doi: 10.18653/v1/2024.emnlp-main.40.
[34] F. Qarah and T. Alsanoosy, “A Comprehensive Analysis of Various Tokenizers for Arabic Large Language Models,” Appl. Sci., vol. 14, no. 13, Jul. 2024, doi: 10.3390/app14135696.
[35] J. Xiao and M. Hong, “A Feature Selection Method Based on a Convolutional Neural Network for Text Classification,” Electron., vol. 14, no. 23, Dec. 2025, doi: 10.3390/electronics14234615.
[36] F. A. D. Aryanti, A. Luthfiarta, and D. A. I. Soeroso, “Aspect-Based Sentiment Analysis with LDA and IndoBERT Algorithm on Mental Health App: Riliv,” J. Appl. Informatics Comput., vol. 9, no. 2, pp. 361–375, 2025, doi: 10.30871/jaic.v9i2.8958.
[37] M. N. Anshory, M. I. Mazdadi, T. H. Saragih, and S. W. Saputro, “Application of Adaboost Algorithm with SMOTE and Optuna Techniques in Sleep Disorder Classification,” Indones. J. Electron. Electromed. Eng. Med. Informatics, no. 2, pp. 415–426, 2025, doi: https://doi.org/10.35882/ijeeemi.v7i2.99.
[38] R. L. Mulianingrum and E. Y. Hidayat, “Comparative Performance of SVM and BERT-Base Using Hybrid Preprocessing for Fast Fashion Sentiment Analysis,” J. Appl. Informatics Comput., vol. 9, no. 6, pp. 3464–3478, 2025.
[39] A. Vaswani, “Attention Is All You Need,” no. Nips, 2017.
[40] M. D. Deepa and A. Tamilarasi, “Bidirectional Encoder Representations from Transformers ( BERT ) Language Model for Sentiment Analysis task : Review,” Turkish J. Comput. Math. Educ., vol. 12, no. 7, pp. 1708–1721, 2021.
[41] A. V. L. Y. A. H. S. Pires, Telmo Pessoa, “One Wide Feedforward is All You Need,” Assoc. Comput. Linguist., vol. https://ac, pp. 1031–1044, 2023, doi: https://doi.org/10.18653/v1/2023.wmt-1.98.
[42] T. L. Models, “Layerwise Importance Analysis of Feed-Forward Networks in Transformer-based Language Models,” Publ. as a Conf. Pap. COLM 2025, no. 2024, pp. 1–21, 2025, doi: https://doi.org/10.48550/arXiv.2508.17734.
[43] H. W. Wibowo, M. Hasbi, and M. Anshori, “Use Discriminant Analysis to Identify Eroticism-Related Terms in The Lyrics of Dangdut Songs,” 2024.
[44] R. Premathilaka, “Context-Aware Detection of Deceptive Design Patterns in E-Commerce Websites Using Word Embedding Based Deep Learning Paradigms,” J. Ilmu Komput. dan Inf. (Journal Comput. Sci. Information), vol. 2, pp. 239–249, 2025.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Eka Mardiana Putri, Mochammad Anshori, M. Syauqi Haris
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
