Comparative Analysis of IndoRoBERTa and EmoRoBERTa for Emotion Classification Based on Ekman’s Taxonomy Using the Indonesian  GoEmotions Dataset

Syifa Naura Milla Celesta; Dinda Lestarini

doi:10.30871/jaic.v10i5.14116

Authors

Syifa Naura Milla Celesta Universitas Sriwijaya
Dinda Lestarini

DOI:

https://doi.org/10.30871/jaic.v10i5.14116

Keywords:

Emotion Classification, EmoRoBERTa, GoEmotions, IndoRoBERTa, Ekman's Taxonomy

Abstract

Emotion classification is an important task in Natural Language Processing (NLP) for identifying emotional information conveyed through Indonesian-language text. However, differences in model specialization may lead to different classification performance, while direct comparisons between language-oriented and emotion-oriented models under the same experimental setting remain limited. This study compares the performance of IndoRoBERTa and EmoRoBERTa for emotion classification using the Indonesian version of the GoEmotions dataset, with mBERT and TF-IDF with SVM included as baseline approaches. The dataset consists of 45,691 single-label samples categorized into seven emotions based on Ekman’s Taxonomy, namely anger, disgust, fear, joy, sadness, surprise, and neutral. The experimental process includes text preprocessing, class weighting to address class imbalance, and hyperparameter tuning to determine the optimal configuration for each model. Model performance is evaluated using accuracy, macro precision, macro recall, and macro F1-score, with macro F1-score as the primary metric. Under the dataset and experimental settings used in this study, IndoRoBERTa obtains an accuracy of 0.6603, macro precision of 0.5938, macro recall of 0.5975, and macro F1-score of 0.5887, while EmoRoBERTa obtains 0.6326, 0.5613, 0.5956, and 0.5742, respectively. mBERT and TF-IDF with SVM obtain macro F1-scores of 0.5495 and 0.5449, respectively. The results provide an empirical comparison of language-oriented and emotion-oriented Transformer models within the experimental setting and provide baseline context for interpreting their classification performance.

Downloads

Download data is not yet available.

References

[1] U. Khairani, V. Mutiawani, and H. Ahmadian, “Pengaruh Tahapan Preprocessing Terhadap Model Indobert Dan Indobertweet Untuk Mendeteksi Emosi Pada Komentar Akun Berita Instagram,” Jurnal Teknologi Informasi dan Ilmu Komputer, vol. 11, no. 4, pp. 887–894, Aug. 2024, doi: 10.25126/jtiik.1148315.

[2] U. Yadav, P. Dasarwar, and D. Asudani, “Integrating commonsense knowledge with GPT embeddings for emotion classification,” MethodsX, vol. 15, Dec. 2025, doi: 10.1016/j.mex.2025.103656.

[3] E. Saraswati and Muljono, “Deteksi Emosi Pada Twitter Berbasis Fasttext: Evaluasi Performa Arsitektur CNN dan GRU,” Rabit : Jurnal Teknologi dan Sistem Informasi Univrab, vol. 11, no. 1, pp. 1175–1186, Jan. 2026, doi: 10.36341/rabit.v11i1.7252.

[4] S. R. Saputro, D. Arifianto, and T. A. Cahyanto, “Klasifikasi Emosi Pada Data Teks Pidato Politik Menggunakan Metode RoBERTa Emotion Classification In Political Speech Texts Using The ROBERTa Model,” Jurnal Teknologi Informasi dan Ilmu Komputer (JTIIK), vol. 13, no. 1, pp. 211–220, Feb. 2026, doi: https://doi.org/10.25126/jtiik.2026131.

[5] D. Demszky, D. Movshovitz-Attias, J. Ko, A. Cowen, G. Nemade, and S. Ravi, “GoEmotions: A Dataset of Fine-Grained Emotions,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Stroudsburg, PA, USA: Association for Computational Linguistics, Jun. 2020, pp. 4040–4054. doi: 10.18653/v1/2020.acl-main.372.

[6] R. Kamath, A. Ghoshal, S. Eswaran, and P. Honnavalli, “An Enhanced Context-based Emotion Detection Model using RoBERTa,” in 2022 IEEE International Conference on Electronics, Computing and Communication Technologies, CONECCT 2022, Institute of Electrical and Electronics Engineers Inc., 2022. doi: 10.1109/CONECCT55679.2022.9865796.

[7] L. Lomasto, N. Lettieri, D. Malandrino, V. Mosca, and R. Zaccagnino, “Modeling emotional signatures to detect conspiratorial communities on social media,” Neural Comput. Appl., vol. 38, no. 8, Apr. 2026, doi: 10.1007/s00521-026-12038-7.

[8] C. M. Greco and A. Tagarelli, “Bringing order into the realm of Transformer-based language models for artificial intelligence and law,” Dec. 01, 2024, Springer Nature. doi: 10.1007/s10506-023-09374-7.

[9] E. Hassan, A. S. Talaat, and M. A. Elsabagh, “Intelligent text similarity assessment using Roberta with integrated chaotic perturbation optimization techniques,” J. Big Data, vol. 12, no. 1, Dec. 2025, doi: 10.1186/s40537-025-01233-3.

[10] A. Araabi, C. Monz, and V. Niculae, “How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation?” [Online]. Available: https://github.com/pytorch/fairseq

[11] W. Suwarningsih, R. A. Pratama, F. Y. Rahadika, and M. H. A. Purnomo, “RoBERTa: language modelling in building Indonesian question-answering systems,” Telkomnika (Telecommunication Computing Electronics and Control), vol. 20, no. 6, pp. 1248–1255, Dec. 2022, doi: 10.12928/TELKOMNIKA.v20i6.24248.

[12] J. R. Putri, Ermatita, and Abdiansah, “Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification,” Journal of Information Systems and Informatics, vol. 8, no. 3, pp. 3595–3622, Jun. 2026, doi: 10.63158/journalisi.v8i3.1664.

[13] M. Dzakwan Bintang Lazuardi and T. Noor Fatyanosa, “Klasifikasi Emosi Multikelas Berbasis Teks Bahasa Indonesia Menggunakan IndoRoBERTa,” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer, vol. 9, no. 9, pp. 2548–964, Sep. 2025, doi: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/15276/6787.

[14] F. Fitriana and H. Setiawan, “Performance Analysis of SVM In Emotion Classification: A Comparative Study Of TF-IDF and Countvectorizer,” Journal of Embedded Systems, Security and Intelligent Systems, pp. 133–145, Jun. 2025, doi: 10.59562/jessi.v6i2.8396.

[15] A. Zamsuri, S. Defit, and G. W. Nurcahyo, “Development and Comparison of Multiple Emotion Classification Models in Indonesia Text Using Machine Learning,” Journal of Advances in Information Technology, vol. 15, no. 4, pp. 519–531, 2024, doi: 10.12720/jait.15.4.519-531.

[16] T. Pires, E. Schlinger, and D. Garrette, “How multilingual is Multilingual BERT?” [Online]. Available: https://github.com/google-research/bert

[17] Y. O. Sihombing, K. Muchlisin, T. F. Arya, Moh. J. Mubarok, and R. F. Rachmadi, “A Systematic Evaluation of BERT Classifiers for Indonesia-based Text Data,” Techno.Com, vol. 25, no. 2, pp. 232–243, May 2026, doi: 10.62411/tc.v25i2.15843.

[18] M. Nurhadi, “Klasifikasi Emosi Multi-Label Pada Teks Berbahasa Indonesia Dengan Fine-Tuning Indobert Menggunakan Dataset GoEmotions,” Sriwijaya University, 2025.

[19] S. Alfariani, O. D. Yanti, S. N. M. Celesta, K. D. Tania, and A. Rifai, “Comparative Analysis of the Performance of Machine Learning Methods and Text Embedding Techniques in Classifying Toxic Conversations in the Roblox Game,” Journal of Applied Informatics and Computing, vol. 10, no. 3, pp. 2775–2789, Jun. 2026, doi: 10.30871/jaic.v10i3.12646.

[20] E. M. Pusung and I. N. Dewi, “Optimasi RoBERTa dengan Hyperparameter Tuning untuk Deteksi Emosi berbasis Teks,” Jurnal Nasional Teknologi dan Sistem Informasi, vol. 10, no. 3, pp. 240–248, Feb. 2025, doi: 10.25077/teknosi.v10i3.2024.240-248.

[21] W. Hardyanto et al., “Improving Sentiment Analysis with a Context-Aware RoBERTa–BiLSTM and Word2Vec Branch,” Scientific Journal of Informatics, vol. 12, no. 4, pp. 785–796, Jan. 2026, doi: 10.15294/sji.v12i4.35918.

[22] A. F. Wibowo and A. Susanto, “Analisis Sentimen Twitter Terhadap Isu Royalti Lagu di Industri Musik Indonesia Menggunakan Naive Bayes dan Support Vector Machine Berbasis TF-IDF,” Technology and Science (BITS), vol. 8, no. 1, 2026, doi: 10.47065/bits.v8i1.9842.

[23] B. Bakirarar and A. H. Elhan, “Class Weighting Technique to Deal with Imbalanced Class Problem in Machine Learning: Methodological Research,” Turkiye Klinikleri Journal of Biostatistics, vol. 15, no. 1, pp. 19–29, 2023, doi: 10.5336/biostatic.2022-93961.

[24] E. Hokijuliandy, H. Napitupulu, and Firdaniza, “Application of SVM and Chi-Square Feature Selection for Sentiment Analysis of Indonesia’s National Health Insurance Mobile Application,” Mathematics, vol. 11, no. 17, Sep. 2023, doi: 10.3390/math11173765.

[25] L. Ke, G. Hu, M. Zhao, Z. Liu, Z. Lv, and Y. Yang, “Brain tumor classification from MRI images using a multi-scale channel attention CNN integrated with SVM,” Sci. Rep., vol. 16, no. 1, Dec. 2026, doi: 10.1038/s41598-026-36164-3.

[26] C. Zhen et al., “Grad-CAM based deep learning analytics for image-level colon disease classification based on graph neural networks and vision transformers,” Front. Physiol., vol. 17, 2026, doi: 10.3389/fphys.2026.1734299.

[27] A. Althunibat, R. Amro, B. Hawashin, H. AlNuhait, S. Almanasra, and H. A. Al-Khawaja, “Automated Classification of User Requirements written in Arabic using machine learning algorithms,” Applied Mathematics and Information Sciences, vol. 17, no. 6, pp. 1155–1170, 2023, doi: 10.18576/AMIS/170620.

[28] Muhammad Mutawakkil Alallah, Mokhammad Amin Hariyadi, Triyo Supriyatno, and E. Riana, “Implementasi BERT dan IndoRoBERTa untuk Klasifikasi Sentimen Opini Publik tentang Kecerdasan Buatan dalam Pendidikan di YouTube,” JURIKOM (Jurnal Riset Komputer), vol. 13, no. 2, pp. 666–678, Apr. 2026, doi: 10.30865/jurikom.v13i2.9697.