Comparing Decision Tree and Optimized LightGBM for Attrition Prediction

Abstract

Employee turnover poses a considerable challenge for organizations, impacting productivity and raising recruitment expenses. This research seeks to evaluate the effectiveness of Decision Tree and Light Gradient Boosting Machine (LightGBM) models in forecasting employee attrition. The study utilizes a quantitative experimental design, leveraging a secondary dataset sourced from Mendeley. Before model development, data preprocessing was performed, and model evaluation was carried out using metrics such as accuracy, precision, recall, F1-score, and ROC-AUC. Each algorithm was assessed under three different configurations baseline, regularization, and hyperparameter tuning through GridSearchCV. The experimental findings indicate that the Decision Tree model is prone to overfitting and has limited capabilities in detecting attrition classes, even though optimization raises the ROC-AUC score to 0.80. In comparison, LightGBM demonstrates more reliable and consistent performance. The Tuned LightGBM model achieved the highest performance on the test dataset, with an Accuracy of 0.81, a Precision of 0.82, a Recall of 0.71, F1-Score of 0.76, and an ROC-AUC of 0.85. An analysis of feature importance reveals that job satisfaction, work-life balance, emotional commitment, work experience, and allowances are the key factors influencing attrition prediction. These results indicate that LightGBM not only performs exceptionally well, but it is also able to offer insights into the critical factors that are important for data-driven retention strategies.