Sentiment Classification for Tabletop Entertainment Discussions Using BERT and VADER Weak-Agreement Rules

Abstract

This research analyzes public sentiment in online tabletop entertainment communities using a hybrid approach that combines lexicon-based and machine-learning methods. Public opinion in the dataset was gathered from Reddit, given its growing reach among hobbyists, and categorized into three sentiment classes: positive, neutral, and negative. The pre-processing stage removes redundant entries, yielding 1494 unique posts. For the model evaluation phase, a relatively balanced testing dataset of 800 entries (500 actual posts and 300 constructed samples) was used, with sentiment distribution of 384 positive, 216 negative, and 200 neutral. Using BERT (bert-base-uncased), the final model achieved an accuracy of 0.8875 and a macro F1-score of 0.8712, with class-level performance as follows: negative (Precision 0.95, Recall 0.95, F1-score 0.95), neutral (Precision 0.97, Recall 0.62, F1-score 0.76), and positive (Precision 0.83, Recall 0.99, F1-score 0.90). The integration of lexicon-based (VADER) and transformer-based (BERT) methods for long-form tabletop community discussions represents the main contribution of this study and can be applied to various sentiment analysis applications. In the exploratory analysis of the actual dataset (1494 posts), sentiment distribution shows dominance of the positive category with 1151 posts (77.04%), followed by 306 neutral (20.48%) and 37 negative (2.48%), indicating real-world class imbalance despite strong balanced-test performance.